Interactive table-based query construction using contextual forms

ABSTRACT

A method includes causing display of events that correspond to search results of a search query in a table. The table includes rows representing events comprising data items of event attributes, columns forming cells with the row, the columns representing respective event attributes, and interactive regions corresponding to one or more data items of the displayed data items. The method also includes in response to the user selecting a designated interactive region, causing display of a list of options, each displayed option corresponding to an interface template for composing query commands, and based on the user selecting an option in the displayed list of options, causing one or more commands to be added to the search query, the one or more commands composed based on the one or more data items that corresponds to the designated interactive region according to instructions of the interface template of the selected option.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/799,917, filed Oct. 31, 2017, and titled “Interactive Table-BasedQuery Construction Using Interface Templates,” which is a continuationof U.S. application Ser. No. 14/815,923, filed Jul. 31, 2015 and nowissued as U.S. Pat. No. 9,836,501, which is a continuation-in-part ofU.S. application Ser. No. 14/611,002, filed Jan. 30, 2015 and now issuedas U.S. Pat. No. 10,061,824, the entire contents of each of theforegoing are hereby incorporated herein by reference in their entirety.

BACKGROUND

Modern data centers often include thousands of hosts that operatecollectively to service requests from even larger numbers of remoteclients. During operation, components of these data centers can producesignificant volumes of machine-generated data. In order to reduce thesize of the data, it is typically pre-processed before it is stored. Insome instances, the pre-processing includes extracting and storing someof the data, but discarding the remainder of the data. Although this maysave storage space in the short term, it can be undesirable in the longterm. For example, if the discarded data is later determined to be ofuse, it may no longer be available.

In some instances, techniques have been developed to apply minimalprocessing to the data in an attempt to preserve more of the data forlater use. For example, the data may be maintained in a relativelyunstructured form to reduce the loss of relevant data. Unfortunately,the unstructured nature of much of this data has made it challenging toperform indexing and searching operations because of the difficulty ofapplying semantic meaning to unstructured data. As the number of hostsand clients associated with a data center continues to grow, processinglarge volumes of machine-generated data in an intelligent manner andeffectively presenting the results of such processing continues to be apriority. Moreover, processing of the data may return a large amount ofinformation that can be difficult for a user to interpret. For example,if a user submits a search of the data, the user may be provided with alarge set of search results for the data but may not know how the searchresults relate to the data itself or how the search results relate toone another. As a result, a user may have a difficult time decipheringwhat portions of the data or the search results are relevant to her/hisinquiry.

SUMMARY

Embodiments of the present invention are directed to Interface templatesfor query commands.

This summary is provided to introduce a selection of concepts in asimplified form that are further described below in the detaileddescription. This summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used in isolation as an aid in determining the scope of the claimedsubject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

Implementations of the present disclosure are described in detail belowwith reference to the attached drawing figures, wherein:

FIG. 1 presents a block diagram of an event-processing system inaccordance with the disclosed embodiments.

FIG. 2 presents a flowchart illustrating how indexers process, index,and store data received from forwarders in accordance with the disclosedembodiments.

FIG. 3 presents a flowchart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments.

FIG. 4 presents a block diagram of a system for processing searchrequests that uses extraction rules for field values in accordance withthe disclosed embodiments.

FIG. 5 illustrates an exemplary search query received from a client andexecuted by search peers in accordance with the disclosed embodiments.

FIG. 6A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 6B illustrates a data summary dialog that enables a user to selectvarious data sources in accordance with the disclosed embodiments.

FIG. 7A illustrates a key indicators view in accordance with thedisclosed embodiments.

FIG. 7B illustrates an incident review dashboard in accordance with thedisclosed embodiments.

FIG. 7C illustrates a proactive monitoring tree in accordance with thedisclosed embodiments.

FIG. 7D illustrates a screen displaying both log data and performancedata in accordance with the disclosed embodiments.

FIG. 8A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8B illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8C illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8D illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8E illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8F illustrates a search screen in accordance with the disclosedembodiments.

FIG. 8G illustrates events in a table format including fields extractedfrom the events.

FIG. 9 illustrates an option menu in accordance with the disclosedembodiments.

FIG. 10 illustrates command entry lists in accordance with the disclosedembodiments.

FIG. 11 illustrates a selection interface in accordance with thedisclosed embodiments.

FIG. 12A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12B illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12C illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12D illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12E illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12F illustrates a search screen in accordance with the disclosedembodiments.

FIG. 12G illustrates a search screen in accordance with the disclosedembodiments.

FIG. 13 illustrates a search screen in accordance with the disclosedembodiments.

FIG. 14A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 14B illustrates a search screen in accordance with the disclosedembodiments.

FIG. 15 illustrates a search screen in accordance with the disclosedembodiments.

FIG. 16 depicts execution of a primary query in accordance with thedisclosed embodiments.

FIG. 17 illustrates a permissions form in accordance with the disclosedembodiments.

FIG. 18A illustrates a permissions interface in accordance with thedisclosed embodiments.

FIG. 18B illustrates a permissions interface in accordance with thedisclosed embodiments.

FIG. 19A illustrates a search screen in accordance with the disclosedembodiments.

FIG. 19B illustrates a data summary view in accordance with thedisclosed embodiments.

FIG. 19C illustrates a data summary view in accordance with thedisclosed embodiments.

FIG. 19D illustrates a data summary view in accordance with thedisclosed embodiments.

FIG. 20 presents a flowchart illustrating utilizing interface templatesfor query commands in accordance with the disclosed embodiments.

FIG. 21 presents a flowchart illustrating adding supplemental eventattributes to a table format in accordance with the disclosedembodiments.

FIG. 22 presents a flowchart illustrating utilizing runtime permissionsfor queries in accordance with the disclosed embodiments.

FIG. 23 presents a flowchart illustrating integrating query interfacesin accordance with the disclosed embodiments.

FIG. 24 presents a flowchart illustrating utilizing a data summary viewin accordance with the disclosed embodiments.

FIG. 25 presents a flowchart illustrating utilizing a data summary viewwith filtering in accordance with the disclosed embodiments.

DETAILED DESCRIPTION

The subject matter of embodiments of the invention is described withspecificity herein to meet statutory requirements. However, thedescription itself is not intended to limit the scope of this patent.Rather, the inventors have contemplated that the claimed subject mattermight be embodied in other ways, to include different steps orcombinations of steps similar to the ones described in this document, inconjunction with other present or future technologies. Moreover,although the terms “step” and/or “block” may be used herein to connotedifferent elements of methods employed, the terms should not beinterpreted as implying any particular order among or between varioussteps herein disclosed unless and except when the order of individualsteps is explicitly described.

1.1 Overview

Modern data centers often comprise thousands of host computer systemsthat operate collectively to service requests from even larger numbersof remote clients. During operation, these data centers generatesignificant volumes of performance data and diagnostic information thatcan be analyzed to quickly diagnose performance problems. In order toreduce the size of this performance data, the data is typicallypre-processed prior to being stored based on anticipated data-analysisneeds. For example, pre-specified data items can be extracted from theperformance data and stored in a database to facilitate efficientretrieval and analysis at search time. However, the rest of theperformance data is not saved and is essentially discarded duringpre-processing. As storage capacity becomes progressively cheaper andmore plentiful, there are fewer incentives to discard this performancedata and many reasons to keep it.

This plentiful storage capacity is presently making it feasible to storemassive quantities of minimally processed performance data at “ingestiontime” for later retrieval and analysis at “search time.” Note thatperforming the analysis operations at search time provides greaterflexibility because it enables an analyst to search all of theperformance data, instead of searching pre-specified data items thatwere stored at ingestion time. This enables the analyst to investigatedifferent aspects of the performance data instead of being confined tothe pre-specified set of data items that were selected at ingestiontime.

However, analyzing massive quantities of heterogeneous performance dataat search time can be a challenging task. A data center may generateheterogeneous performance data from thousands of different components,which can collectively generate tremendous volumes of performance datathat can be time-consuming to analyze. For example, this performancedata can include data from system logs, network packet data, sensordata, and data generated by various applications. Also, the unstructurednature of much of this performance data can pose additional challengesbecause of the difficulty of applying semantic meaning to unstructureddata, and the difficulty of indexing and querying unstructured datausing traditional database systems.

These challenges can be addressed by using an event-based system, suchas the SPLUNK® ENTERPRISE system produced by Splunk Inc. of SanFrancisco, Calif., to store and process performance data. The SPLUNK®ENTERPRISE system is the leading platform for providing real-timeoperational intelligence that enables organizations to collect, index,and harness machine-generated data from various websites, applications,servers, networks, and mobile devices that power their businesses. TheSPLUNK® ENTERPRISE system is particularly useful for analyzingunstructured performance data, which is commonly found in system logfiles. Although many of the techniques described herein are explainedwith reference to the SPLUNK® ENTERPRISE system, the techniques are alsoapplicable to other types of data server systems.

In the SPLUNK® ENTERPRISE system, performance data is stored as“events,” wherein each event comprises a collection of performance dataand/or diagnostic information that is generated by a computer system andis correlated with a specific point in time. Events can be derived from“time series data,” wherein time series data comprises a sequence ofdata points (e.g., performance measurements from a computer system) thatare associated with successive points in time and are typically spacedat uniform time intervals. Events can also be derived from “structured”or “unstructured” data. Structured data has a predefined format, whereinspecific data items with specific data formats reside at predefinedlocations in the data. For example, structured data can include dataitems stored in fields in a database table. In contrast, unstructureddata does not have a predefined format. This means that unstructureddata can comprise various data items having different data types thatcan reside at different locations. For example, when the data source isan operating system log, an event can include one or more lines from theoperating system log containing raw data that includes different typesof performance and diagnostic information associated with a specificpoint in time. Examples of data sources from which an event may bederived include, but are not limited to: web servers; applicationservers; databases; firewalls; routers; operating systems; and softwareapplications that execute on computer systems, mobile devices, andsensors. The data generated by such data sources can be produced invarious forms including, for example and without limitation, server logfiles, activity log files, configuration files, messages, network packetdata, performance measurements, and sensor measurements. An eventtypically includes a timestamp that may be derived from the raw data inthe event, or may be determined through interpolation between temporallyproximate events having known timestamps.

The SPLUNK® ENTERPRISE system also facilitates using a flexible schemato specify how to extract information from the event data, wherein theflexible schema may be developed and redefined as needed. Note that aflexible schema may be applied to event data “on the fly,” when it isneeded (e.g., at search time), rather than at ingestion time of the dataas in traditional database systems. Because the schema is not applied toevent data until it is needed (e.g., at search time), it is referred toas a “late-binding schema.”

During operation, the SPLUNK® ENTERPRISE system starts with raw data,which can include unstructured data, machine data, performancemeasurements, or other time-series data, such as data obtained fromweblogs, syslogs, or sensor readings. It divides this raw data into“portions,” and optionally transforms the data to produce timestampedevents. The system stores the timestamped events in a data store, andenables a user to run queries against the data store to retrieve eventsthat meet specified criteria, such as containing certain keywords orhaving specific values in defined fields. Note that the term “field”refers to a location in the event data containing a value for a specificdata item.

As noted above, the SPLUNK® ENTERPRISE system facilitates using alate-binding schema while performing queries on events. A late-bindingschema specifies “extraction rules” that are applied to data in theevents to extract values for specific fields. More specifically, theextraction rules for a field can include one or more instructions thatspecify how to extract a value for the field from the event data. Anextraction rule can generally include any type of instruction forextracting values from data in events. In some cases, an extraction rulecomprises a regular expression, in which case the rule is referred to asa “regex rule.”

In contrast to a conventional schema for a database system, alate-binding schema is not defined at data ingestion time. Instead, thelate-binding schema can be developed on an ongoing basis until the timea query is actually executed. This means that extraction rules for thefields in a query may be provided in the query itself, or may be locatedduring execution of the query. Hence, as an analyst learns more aboutthe data in the events, the analyst can continue to refine thelate-binding schema by adding new fields, deleting fields, or changingthe field extraction rules until the next time the schema is used by aquery. Because the SPLUNK® ENTERPRISE system maintains the underlyingraw data and provides a late-binding schema for searching the raw data,it enables an analyst to investigate questions that arise as the analystlearns more about the events.

In the SPLUNK® ENTERPRISE system, a field extractor may be configured toautomatically generate extraction rules for certain fields in the eventswhen the events are being created, indexed, or stored, or possibly at alater time. Alternatively, a user may manually define extraction rulesfor fields using a variety of techniques.

Also, a number of “default fields” that specify metadata about theevents rather than data in the events themselves can be createdautomatically. For example, such default fields can specify: a timestampfor the event data; a host from which the event data originated; asource of the event data; and a source type for the event data. Thesedefault fields may be determined automatically when the events arecreated, indexed or stored.

In some embodiments, a common field name may be used to reference two ormore fields containing equivalent data items, even though the fields maybe associated with different types of events that possibly havedifferent data formats and different extraction rules. By enabling acommon field name to be used to identify equivalent fields fromdifferent types of events generated by different data sources, thesystem facilitates use of a “common information model” (CIM) across thedifferent data sources.

1.2 Data Server System

FIG. 1 presents a block diagram of an exemplary event-processing system100, similar to the SPLUNK® ENTERPRISE system. System 100 includes oneor more forwarders 101 that collect data obtained from a variety ofdifferent data sources 105, and one or more indexers 102 that store,process, and/or perform operations on this data, wherein each indexeroperates on data contained in a specific data store 103. Theseforwarders and indexers can comprise separate computer systems in a datacenter, or may alternatively comprise separate processes executing onvarious computer systems in a data center.

During operation, the forwarders 101 identify which indexers 102 willreceive the collected data and then forward the data to the identifiedindexers. Forwarders 101 can also perform operations to strip outextraneous data and detect timestamps in the data. The forwarders nextdetermine which indexers 102 will receive each data item and thenforward the data items to the determined indexers 102.

Note that distributing data across different indexers facilitatesparallel processing. This parallel processing can take place at dataingestion time, because multiple indexers can process the incoming datain parallel. The parallel processing can also take place at search time,because multiple indexers can search through the data in parallel.

System 100 and the processes described below with respect to FIGS. 1-5are further described in “Exploring Splunk Search Processing Language(SPL) Primer and Cookbook” by David Carasso, CITO Research, 2012, and in“Optimizing Data Analysis With a Semi-Structured Time Series Database”by Ledion Bitincka, Archana Ganapathi, Stephen Sorkin, and Steve Zhang,SLAML, 2010, each of which is hereby incorporated herein by reference inits entirety for all purposes.

1.3 Data Ingestion

FIG. 2 presents a flowchart illustrating how an indexer processes,indexes, and stores data received from forwarders in accordance with thedisclosed embodiments. At block 201, the indexer receives the data fromthe forwarder. Next, at block 202, the indexer apportions the data intoevents. Note that the data can include lines of text that are separatedby carriage returns or line breaks and an event may include one or moreof these lines. During the apportioning process, the indexer can useheuristic rules to automatically determine the boundaries of the events,which for example coincide with line boundaries. These heuristic rulesmay be determined based on the source of the data, wherein the indexercan be explicitly informed about the source of the data or can infer thesource of the data by examining the data. These heuristic rules caninclude regular expression-based rules or delimiter-based rules fordetermining event boundaries, wherein the event boundaries may beindicated by predefined characters or character strings. Thesepredefined characters may include punctuation marks or other specialcharacters including, for example, carriage returns, tabs, spaces orline breaks. In some cases, a user can fine-tune or configure the rulesthat the indexers use to determine event boundaries in order to adaptthe rules to the user's specific requirements.

Next, the indexer determines a timestamp for each event at block 203. Asmentioned above, these timestamps can be determined by extracting thetime directly from data in the event, or by interpolating the time basedon timestamps from temporally proximate events. In some cases, atimestamp can be determined based on the time the data was received orgenerated. The indexer subsequently associates the determined timestampwith each event at block 204, for example by storing the timestamp asmetadata for each event.

Then, the system can apply transformations to data to be included inevents at block 205. For log data, such transformations can includeremoving a portion of an event (e.g., a portion used to define eventboundaries, extraneous text, characters, etc.) or removing redundantportions of an event. Note that a user can specify portions to beremoved using a regular expression or any other possible technique.

Next, a keyword index can optionally be generated to facilitate fastkeyword searching for events. To build a keyword index, the indexerfirst identifies a set of keywords in block 206. Then, at block 207 theindexer includes the identified keywords in an index, which associateseach stored keyword with references to events containing that keyword(or to locations within events where that keyword is located). When anindexer subsequently receives a keyword-based query, the indexer canaccess the keyword index to quickly identify events containing thekeyword.

In some embodiments, the keyword index may include entries forname-value pairs found in events, wherein a name-value pair can includea pair of keywords connected by a symbol, such as an equals sign orcolon. In this way, events containing these name-value pairs can bequickly located. In some embodiments, fields can automatically begenerated for some or all of the name-value pairs at the time ofindexing. For example, if the string “dest=10.0.1.2” is found in anevent, a field named “dest” may be created for the event, and assigned avalue of “10.0.1.2.”

Finally, the indexer stores the events in a data store at block 208,wherein a timestamp can be stored with each event to facilitatesearching for events based on a time range. In some cases, the storedevents are organized into a plurality of buckets, wherein each bucketstores events associated with a specific time range. This not onlyimproves time based searches, but it also allows events with recenttimestamps that may have a higher likelihood of being accessed to bestored in faster memory to facilitate faster retrieval. For example, abucket containing the most recent events can be stored in flash memoryinstead of on hard disk.

Each indexer 102 is responsible for storing and searching a subset ofthe events contained in a corresponding data store 103. By distributingevents among the indexers and data stores, the indexers can analyzeevents for a query in parallel, for example using map-reduce techniques,wherein each indexer returns partial responses for a subset of events toa search head that combines the results to produce an answer for thequery. By storing events in buckets for specific time ranges, an indexermay further optimize searching by looking only in buckets for timeranges that are relevant to a query.

Moreover, events and buckets can also be replicated across differentindexers and data stores to facilitate high availability and disasterrecovery as is described in U.S. patent application Ser. No. 14/266,812filed on 30 Apr. 2014, and in U.S. patent application Ser. No.14/266,817 also filed on 30 Apr. 2014.

1.4 Query Processing

FIG. 3 presents a flowchart illustrating how a search head and indexersperform a search query in accordance with the disclosed embodiments. Atthe start of this process, a search head receives a search query from aclient at block 301. Next, at block 302, the search head analyzes thesearch query to determine what portions can be delegated to indexers andwhat portions need to be executed locally by the search head. At block303, the search head distributes the determined portions of the query tothe indexers. Note that commands that operate on single events can betrivially delegated to the indexers, while commands that involve eventsfrom multiple indexers are harder to delegate.

Then, at block 304, the indexers to which the query was distributedsearch their data stores for events that are responsive to the query. Todetermine which events are responsive to the query, the indexer searchesfor events that match the criteria specified in the query. This criteriacan include matching keywords or specific values for certain fields. Ina query that uses a late-binding schema, the searching operations inblock 304 may involve using the late-binding scheme to extract valuesfor specified fields from events at the time the query is processed.Next, the indexers can either send the relevant events back to thesearch head, or use the events to calculate a partial result, and sendthe partial result back to the search head.

Finally, at block 305, the search head combines the partial resultsand/or events received from the indexers to produce a final result forthe query. This final result can comprise different types of datadepending upon what the query is asking for. For example, the finalresults can include a listing of matching events returned by the query,or some type of visualization of data from the returned events. Inanother example, the final result can include one or more calculatedvalues derived from the matching events.

Moreover, the results generated by system 100 can be returned to aclient using different techniques. For example, one technique streamsresults back to a client in real-time as they are identified. Anothertechnique waits to report results to the client until a complete set ofresults is ready to return to the client. Yet another technique streamsinterim results back to the client in real-time until a complete set ofresults is ready, and then returns the complete set of results to theclient. In another technique, certain results are stored as “searchjobs,” and the client may subsequently retrieve the results byreferencing the search jobs.

The search head can also perform various operations to make the searchmore efficient. For example, before the search head starts executing aquery, the search head can determine a time range for the query and aset of common keywords that all matching events must include. Next, thesearch head can use these parameters to query the indexers to obtain asuperset of the eventual results. Then, during a filtering stage, thesearch head can perform field-extraction operations on the superset toproduce a reduced set of search results.

1.5 Field Extraction

FIG. 4 presents a block diagram illustrating how fields can be extractedduring query processing in accordance with the disclosed embodiments. Atthe start of this process, a search query 402 is received at a queryprocessor 404. Query processor 404 includes various mechanisms forprocessing a query, wherein these mechanisms can reside in a search head104 and/or an indexer 102. Note that the exemplary search query 402illustrated in FIG. 4 is expressed in Search Processing Language (SPL),which is used in conjunction with the SPLUNK® ENTERPRISE system. SPL isa pipelined search language in which a set of inputs is operated on by afirst command in a command line, and then a subsequent command followingthe pipe symbol “1” operates on the results produced by the firstcommand, and so on for additional commands. Search query 402 can also beexpressed in other query languages, such as the Structured QueryLanguage (“SQL”) or any suitable query language.

Upon receiving search query 402, query processor 404 sees that searchquery 402 includes two fields “IP” and “target.” Query processor 404also determines that the values for the “IP” and “target” fields havenot already been extracted from events in data store 414, andconsequently determines that query processor 404 needs to use extractionrules to extract values for the fields. Hence, query processor 404performs a lookup for the extraction rules in a rule base 406, whereinrule base 406 maps field names to corresponding extraction rules andobtains extraction rules 408-409, wherein extraction rule 408 specifieshow to extract a value for the “IP” field from an event, and extractionrule 409 specifies how to extract a value for the “target” field from anevent. As is illustrated in FIG. 4, extraction rules 408-409 cancomprise regular expressions that specify how to extract values for therelevant fields. Such regular-expression-based extraction rules are alsoreferred to as “regex rules.” In addition to specifying how to extractfield values, the extraction rules may also include instructions forderiving a field value by performing a function on a character string orvalue retrieved by the extraction rule. For example, a transformationrule may truncate a character string, or convert the character stringinto a different data format. In some cases, the query itself canspecify one or more extraction rules.

Next, query processor 404 sends extraction rules 408-409 to a fieldextractor 412, which applies extraction rules 408-409 to events 416-418in a data store 414. Note that data store 414 can include one or moredata stores, and extraction rules 408-409 can be applied to largenumbers of events in data store 414, and are not meant to be limited tothe three events 416-418 illustrated in FIG. 4. Moreover, the queryprocessor 404 can instruct field extractor 412 to apply the extractionrules to all the events in a data store 414, or to a subset of theevents that have been filtered based on some criteria.

Next, field extractor 412 applies extraction rule 408 for the firstcommand “Search IP=“10*” to events in data store 414 including events416-418. Extraction rule 408 is used to extract values for the IPaddress field from events in data store 414 by looking for a pattern ofone or more digits, followed by a period, followed again by one or moredigits, followed by another period, followed again by one or moredigits, followed by another period, and followed again by one or moredigits. Next, field extractor 412 returns field values 420 to queryprocessor 404, which uses the criterion IP=“10*” to look for IPaddresses that start with “10”. Note that events 416 and 417 match thiscriterion, but event 418 does not, so the result set for the firstcommand is events 416-417.

Query processor 404 then sends events 416-417 to the next command “statscount target.” To process this command, query processor 404 causes fieldextractor 412 to apply extraction rule 409 to events 416-417. Extractionrule 409 is used to extract values for the target field for events416-417 by skipping the first four commas in events 416-417, and thenextracting all of the following characters until a comma or period isreached. Next, field extractor 412 returns field values 421 to queryprocessor 404, which executes the command “stats count target” to countthe number of unique values contained in the target fields, which inthis example produces the value “2” that is returned as a final result422 for the query.

Note that query results can be returned to a client, a search head, orany other system component for further processing. In general, queryresults may include: a set of one or more events; a set of one or morevalues obtained from the events; a subset of the values; statisticscalculated based on the values; a report containing the values; or avisualization, such as a graph or chart, generated from the values.

1.6 Exemplary Search Screen

FIG. 6A illustrates an exemplary search screen 600 in accordance withthe disclosed embodiments. Search screen 600 includes a search bar 602that accepts user input in the form of a search string. It also includesa time range picker 612 that enables the user to specify a time rangefor the search. For “historical searches” the user can select a specifictime range, or alternatively a relative time range, such as “today,”“yesterday” or “last week.” For “real-time searches,” the user canselect the size of a preceding time window to search for real-timeevents. Search screen 600 also initially displays a “data summary”dialog as is illustrated in FIG. 6B that enables the user to selectdifferent sources for the event data, for example by selecting specifichosts and log files.

After the search is executed, the search screen 600 can display theresults through search results tabs 604, wherein search results tabs 604include: an “events tab” that displays various information about eventsreturned by the search; a “statistics tab” that displays statisticsabout the search results; and a “visualization tab” that displaysvarious visualizations of the search results. The events tab illustratedin FIG. 6A displays a timeline graph 605 that graphically illustratesthe number of events that occurred in one-hour intervals over theselected time range. It also displays an events list 608 that enables auser to view the raw data in each of the returned events. Itadditionally displays a fields sidebar 606 that includes statisticsabout occurrences of specific fields in the returned events, including“selected fields” that are pre-selected by the user, and “interestingfields” that are automatically selected by the system based onpre-specified criteria.

1.7 Acceleration Techniques

The above-described system provides significant flexibility by enablinga user to analyze massive quantities of minimally processed performancedata “on the fly” at search time instead of storing pre-specifiedportions of the performance data in a database at ingestion time. Thisflexibility enables a user to see correlations in the performance dataand perform subsequent queries to examine interesting aspects of theperformance data that may not have been apparent at ingestion time.

However, performing extraction and analysis operations at search timecan involve a large amount of data and require a large number ofcomputational operations, which can cause considerable delays whileprocessing the queries. Fortunately, a number of acceleration techniqueshave been developed to speed up analysis operations performed at searchtime. These techniques include: (1) performing search operations inparallel by formulating a search as a map-reduce computation; (2) usinga keyword index; (3) using a high performance analytics store; and (4)accelerating the process of generating reports. These techniques aredescribed in more detail below.

1.7.1 Map-Reduce Technique

To facilitate faster query processing, a query can be structured as amap-reduce computation, wherein the “map” operations are delegated tothe indexers, while the corresponding “reduce” operations are performedlocally at the search head. For example, FIG. 5 illustrates how a searchquery 501 received from a client at search head 104 can be split intotwo phases, including: (1) a “map phase” comprising subtasks 502 (e.g.,data retrieval or simple filtering) that may be performed in paralleland are “mapped” to indexers 102 for execution, and (2) a “reduce phase”comprising a merging operation 503 to be executed by the search headwhen the results are ultimately collected from the indexers.

During operation, upon receiving search query 501, search head 104modifies search query 501 by substituting “stats” with “prestats” toproduce search query 502, and then distributes search query 502 to oneor more distributed indexers, which are also referred to as “searchpeers.” Note that search queries may generally specify search criteriaor operations to be performed on events that meet the search criteria.Search queries may also specify field names, as well as search criteriafor the values in the fields or operations to be performed on the valuesin the fields. Moreover, the search head may distribute the full searchquery to the search peers as is illustrated in FIG. 3, or mayalternatively distribute a modified version (e.g., a more restrictedversion) of the search query to the search peers. In this example, theindexers are responsible for producing the results and sending them tothe search head. After the indexers return the results to the searchhead, the search head performs the merging operations 503 on theresults. Note that by executing the computation in this way, the systemeffectively distributes the computational operations while minimizingdata transfers.

1.7.2 Keyword Index

As described above with reference to the flow charts in FIGS. 2 and 3,event-processing system 100 can construct and maintain one or morekeyword indices to facilitate rapidly identifying events containingspecific keywords. This can greatly speed up the processing of queriesinvolving specific keywords. As mentioned above, to build a keywordindex, an indexer first identifies a set of keywords. Then, the indexerincludes the identified keywords in an index, which associates eachstored keyword with references to events containing that keyword, or tolocations within events where that keyword is located. When an indexersubsequently receives a keyword-based query, the indexer can access thekeyword index to quickly identify events containing the keyword.

1.7.3 High Performance Analytics Store

To speed up certain types of queries, some embodiments of system 100make use of a high performance analytics store, which is referred to asa “summarization table,” that contains entries for specific field-valuepairs. Each of these entries keeps track of instances of a specificvalue in a specific field in the event data and includes references toevents containing the specific value in the specific field. For example,an exemplary entry in a summarization table can keep track ofoccurrences of the value “94107” in a “ZIP code” field of a set ofevents, wherein the entry includes references to all of the events thatcontain the value “94107” in the ZIP code field. This enables the systemto quickly process queries that seek to determine how many events have aparticular value for a particular field, because the system can examinethe entry in the summarization table to count instances of the specificvalue in the field without having to go through the individual events ordo extractions at search time. Also, if the system needs to process allevents that have a specific field-value combination, the system can usethe references in the summarization table entry to directly access theevents to extract further information without having to search all ofthe events to find the specific field-value combination at search time.

In some embodiments, the system maintains a separate summarization tablefor each of the above-described time-specific buckets that stores eventsfor a specific time range, wherein a bucket-specific summarization tableincludes entries for specific field-value combinations that occur inevents in the specific bucket. Alternatively, the system can maintain aseparate summarization table for each indexer, wherein theindexer-specific summarization table only includes entries for theevents in a data store that is managed by the specific indexer.

The summarization table can be populated by running a “collection query”that scans a set of events to find instances of a specific field-valuecombination, or alternatively instances of all field-value combinationsfor a specific field. A collection query can be initiated by a user, orcan be scheduled to occur automatically at specific time intervals. Acollection query can also be automatically launched in response to aquery that asks for a specific field-value combination.

In some cases, the summarization tables may not cover all of the eventsthat are relevant to a query. In this case, the system can use thesummarization tables to obtain partial results for the events that arecovered by summarization tables, but may also have to search throughother events that are not covered by the summarization tables to produceadditional results. These additional results can then be combined withthe partial results to produce a final set of results for the query.This summarization table and associated techniques are described in moredetail in U.S. Pat. No. 8,682,925, issued on Mar. 25, 2014.

1.7.4 Accelerating Report Generation

In some embodiments, a data server system such as the SPLUNK® ENTERPRISEsystem can accelerate the process of periodically generating updatedreports based on query results. To accelerate this process, asummarization engine automatically examines the query to determinewhether generation of updated reports can be accelerated by creatingintermediate summaries (This is possible if results from preceding timeperiods can be computed separately and combined to generate an updatedreport. In some cases, it is not possible to combine such incrementalresults, for example where a value in the report depends onrelationships between events from different time periods.) If reportscan be accelerated, the summarization engine periodically generates asummary covering data obtained during a latest non-overlapping timeperiod. For example, where the query seeks events meeting a specifiedcriteria, a summary for the time period includes only events within thetime period that meet the specified criteria. Similarly, if the queryseeks statistics calculated from the events, such as the number ofevents that match the specified criteria, then the summary for the timeperiod includes the number of events in the period that match thespecified criteria.

In parallel with the creation of the summaries, the summarization engineschedules the periodic updating of the report associated with the query.During each scheduled report update, the query engine determines whetherintermediate summaries have been generated covering portions of the timeperiod covered by the report update. If so, then the report is generatedbased on the information contained in the summaries. Also, if additionalevent data has been received and has not yet been summarized, and isrequired to generate the complete report, the query can be run on thisadditional event data. Then, the results returned by this query on theadditional event data, along with the partial results obtained from theintermediate summaries, can be combined to generate the updated report.This process is repeated each time the report is updated. Alternatively,if the system stores events in buckets covering specific time ranges,then the summaries can be generated on a bucket-by-bucket basis. Notethat producing intermediate summaries can save the work involved inre-running the query for previous time periods, so only the newer eventdata needs to be processed while generating an updated report. Thesereport acceleration techniques are described in more detail in U.S. Pat.No. 8,589,403, issued on Nov. 19, 2013, and U.S. Pat. No. 8,412,696,issued on Apr. 2, 2011.

1.8 Security Features

The SPLUNK® ENTERPRISE platform provides various schemas, dashboards andvisualizations that make it easy for developers to create applicationsto provide additional capabilities. One such application is the SPLUNK®APP FOR ENTERPRISE SECURITY, which performs monitoring and alertingoperations and includes analytics to facilitate identifying both knownand unknown security threats based on large volumes of data stored bythe SPLUNK® ENTERPRISE system. This differs significantly fromconventional Security Information and Event Management (SIEM) systemsthat lack the infrastructure to effectively store and analyze largevolumes of security-related event data. Traditional SIEM systemstypically use fixed schemas to extract data from pre-definedsecurity-related fields at data ingestion time, wherein the extracteddata is typically stored in a relational database. This data extractionprocess (and associated reduction in data size) that occurs at dataingestion time inevitably hampers future incident investigations, whenall of the original data may be needed to determine the root cause of asecurity issue, or to detect the tiny fingerprints of an impendingsecurity threat.

In contrast, the SPLUNK® APP FOR ENTERPRISE SECURITY system stores largevolumes of minimally processed security-related data at ingestion timefor later retrieval and analysis at search time when a live securitythreat is being investigated. To facilitate this data retrieval process,the SPLUNK® APP FOR ENTERPRISE SECURITY provides pre-specified schemasfor extracting relevant values from the different types ofsecurity-related event data, and also enables a user to define suchschemas.

The SPLUNK® APP FOR ENTERPRISE SECURITY can process many types ofsecurity-related information. In general, this security-relatedinformation can include any information that can be used to identifysecurity threats. For example, the security-related information caninclude network-related information, such as IP addresses, domain names,asset identifiers, network traffic volume, uniform resource locatorstrings, and source addresses. (The process of detecting securitythreats for network-related information is further described in U.S.patent application Ser. Nos. 13/956,252, and 13/956,262.)Security-related information can also include endpoint information, suchas malware infection data and system configuration information, as wellas access control information, such as login/logout information andaccess failure notifications. The security-related information canoriginate from various sources within a data center, such as hosts,virtual machines, storage devices, and sensors. The security-relatedinformation can also originate from various sources in a network, suchas routers, switches, email servers, proxy servers, gateways, firewalls,and intrusion-detection systems.

During operation, the SPLUNK® APP FOR ENTERPRISE SECURITY facilitatesdetecting so-called “notable events” that are likely to indicate asecurity threat. These notable events can be detected in a number ofways: (1) an analyst can notice a correlation in the data and canmanually identify a corresponding group of one or more events as“notable;” or (2) an analyst can define a “correlation search”specifying criteria for a notable event, and every time one or moreevents satisfy the criteria, the application can indicate that the oneor more events are notable. An analyst can alternatively select apre-defined correlation search provided by the application. Note thatcorrelation searches can be run continuously or at regular intervals(e.g., every hour) to search for notable events. Upon detection, notableevents can be stored in a dedicated “notable events index,” which can besubsequently accessed to generate various visualizations containingsecurity-related information. Also, alerts can be generated to notifysystem operators when important notable events are discovered.

The SPLUNK® APP FOR ENTERPRISE SECURITY provides various visualizationsto aid in discovering security threats, such as a “key indicators view”that enables a user to view security metrics of interest, such as countsof different types of notable events. For example, FIG. 7A illustratesan exemplary key indicators view 700 that comprises a dashboard, whichcan display a value 701, for various security-related metrics, such asmalware infections 702. It can also display a change in a metric value703, which indicates that the number of malware infections increased by63 during the preceding interval. Key indicators view 700 additionallydisplays a histogram panel 704 that displays a histogram of notableevents organized by urgency values, and a histogram of notable eventsorganized by time intervals. This key indicators view is described infurther detail in pending U.S. patent application Ser. No. 13/956,338filed Jul. 31, 2013.

These visualizations can also include an “incident review dashboard”that enables a user to view and act on “notable events.” These notableevents can include: (1) a single event of high importance, such as anyactivity from a known web attacker; or (2) multiple events thatcollectively warrant review, such as a large number of authenticationfailures on a host followed by a successful authentication. For example,FIG. 7B illustrates an exemplary incident review dashboard 710 thatincludes a set of incident attribute fields 711 that, for example,enables a user to specify a time range field 712 for the displayedevents. It also includes a timeline 713 that graphically illustrates thenumber of incidents that occurred in one-hour time intervals over theselected time range. It additionally displays an events list 714 thatenables a user to view a list of all of the notable events that matchthe criteria in the incident attributes fields 711. To facilitateidentifying patterns among the notable events, each notable event can beassociated with an urgency value (e.g., low, medium, high, critical),which is indicated in the incident review dashboard. The urgency valuefor a detected event can be determined based on the severity of theevent and the priority of the system component associated with theevent. The incident review dashboard is described further in“http://docs.splunk.com/Documentation/PCI/2.1.1/User/IncidentReviewdashboard.”

1.9 Data Center Monitoring

As mentioned above, the SPLUNK® ENTERPRISE platform provides variousfeatures that make it easy for developers to create variousapplications. One such application is the SPLUNK® APP FOR VMWARE®, whichperforms monitoring operations and includes analytics to facilitatediagnosing the root cause of performance problems in a data center basedon large volumes of data stored by the SPLUNK® ENTERPRISE system.

This differs from conventional data-center-monitoring systems that lackthe infrastructure to effectively store and analyze large volumes ofperformance information and log data obtained from the data center. Inconventional data-center-monitoring systems, this performance data istypically pre-processed prior to being stored, for example by extractingpre-specified data items from the performance data and storing them in adatabase to facilitate subsequent retrieval and analysis at search time.However, the rest of the performance data is not saved and isessentially discarded during pre-processing. In contrast, the SPLUNK®APP FOR VMWARE® stores large volumes of minimally processed performanceinformation and log data at ingestion time for later retrieval andanalysis at search time when a live performance issue is beinginvestigated.

The SPLUNK® APP FOR VMWARE® can process many types ofperformance-related information. In general, this performance-relatedinformation can include any type of performance-related data and logdata produced by virtual machines and host computer systems in a datacenter. In addition to data obtained from various log files, thisperformance-related information can include values for performancemetrics obtained through an application programming interface (API)provided as part of the vSphere Hypervisor™ system distributed byVMware, Inc. of Palo Alto, Calif. For example, these performance metricscan include: (1) CPU-related performance metrics; (2) disk-relatedperformance metrics; (3) memory-related performance metrics; (4)network-related performance metrics; (5) energy-usage statistics; (6)data-traffic-related performance metrics; (7) overall systemavailability performance metrics; (8) cluster-related performancemetrics; and (9) virtual machine performance statistics. For moredetails about such performance metrics, please see U.S. patentapplication Ser. No. 14/167,316 filed 29 Jan. 2014, which is herebyincorporated herein by reference. Also, see “vSphere Monitoring andPerformance,” Update 1, vSphere 5.5, EN-001357-00,http://pubs.vmware.com/vsphere-55/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-551-monitoring-performance-guide.pdf.

To facilitate retrieving information of interest from performance dataand log files, the SPLUNK® APP FOR VMWARE® provides pre-specifiedschemas for extracting relevant values from different types ofperformance-related event data, and also enables a user to define suchschemas.

The SPLUNK® APP FOR VMWARE® additionally provides various visualizationsto facilitate detecting and diagnosing the root cause of performanceproblems. For example, one such visualization is a “proactive monitoringtree” that enables a user to easily view and understand relationshipsamong various factors that affect the performance of a hierarchicallystructured computing system. This proactive monitoring tree enables auser to easily navigate the hierarchy by selectively expanding nodesrepresenting various entities (e.g., virtual centers or computingclusters) to view performance information for lower-level nodesassociated with lower-level entities (e.g., virtual machines or hostsystems). Exemplary node-expansion operations are illustrated in FIG.7C, wherein nodes 733 and 734 are selectively expanded. Note that nodes731-739 can be displayed using different patterns or colors to representdifferent performance states, such as a critical state, a warning state,a normal state, or an unknown/offline state. The ease of navigationprovided by selective expansion in combination with the associatedperformance-state information enables a user to quickly diagnose theroot cause of a performance problem. The proactive monitoring tree isdescribed in further detail in U.S. patent application Ser. No.14/235,490 filed on 15 Apr. 2014, which is hereby incorporated herein byreference for all possible purposes.

The SPLUNK® APP FOR VMWARE® also provides a user interface that enablesa user to select a specific time range and then view heterogeneous data,comprising events, log data, and associated performance metrics, for theselected time range. For example, the screen illustrated in FIG. 7Ddisplays a listing of recent “tasks and events” and a listing of recent“log entries” for a selected time range above a performance-metric graphfor “average CPU core utilization” for the selected time range. Notethat a user is able to operate pull-down menus 742 to selectivelydisplay different performance metric graphs for the selected time range.This enables the user to correlate trends in the performance-metricgraph with corresponding event and log data to quickly determine theroot cause of a performance problem. This user interface is described inmore detail in U.S. patent application Ser. No. 14/167,316 filed on 29Jan. 2014, which is hereby incorporated herein by reference for allpossible purposes.

2.0 Additional Exemplary Search Screen

FIG. 8A illustrates exemplary search screen 800 in accordance with someimplementations of the present disclosure. Search screen 800 may beutilized as part of a search interface to display one or more eventsreturned as part of a search result set based on a search query. Displayof an event may include display of one or more event attributes of theevent, examples of which include extracted fields, metadata, event rawdata, and/or other types of data items assigned to the event. Searchscreen 800 may also be utilized as part of a search interface thatallows a user to modify the search query. Some exemplary options formodifying the search query include any combination of deleting commandsfrom the search query, adding commands to the search query, reorderingone or more commands in the search query, and modifying variables,parameters, arguments, and/or other properties of commands in the searchquery.

Search screen 800 may also be utilized to update the search result setto correspond to the modified search query and to update the events thatare displayed in search screen 800 to correspond to the updated searchresult set. In some cases, based on a search query being modified, thesearch query could be completely re-executed to retrieve new searchresults and generate the updated search result set. In other cases, thesearch query may only be partially executed. For example, inimplementations where a pipelined search language, such as SPL, isemployed for search queries, additional commands that are added to asearch query may be applied to at least some previous search results.These and other variations are possible for updating the search resultset to correspond to a search query.

By interacting with search screen 800 to create and/or modify searchqueries, a user may utilize the search interface to filter, sort, clean,enrich, analyze, report on, and/or otherwise carryout functionalityprovided for by commands in search queries. Furthermore, as the usergenerates modified search queries, the search result set can be updated,with events displayed in search screen 800 being updated to reflect themodifications. Utilizing this approach, a user may iteratively modify asearch query and view the impact of the modification via updated searchresults. This approach can be employed to enable users to effectivelyand efficiently generate queries that return expected and desiredresults, even without extensive knowledge of the underlying commandsand/or search language employed by the queries.

2.1 Exemplary Table Format

In various implementations, one or more events can be displayed in atable format, such as table format 802 in search screen 800. The tableformat can be employed in various interfaces for interacting withdisplayed events in various ways and its use is not limited to searchinterfaces or search screens. Events can be used to populate the tableformat, and may be search results, such as in search screen 800, butcould more generally be any type of events. Furthermore while in searchscreen 800, events are displayed via table format 802, other formats arepossible in various implementations contemplated herein.

Table format 802 comprises one or more columns, such as columns 804 a,804 b, 804 c, and 804 d and one or more rows, such as rows 806 a, 806 b,806 c, 806 d, and 806 e. Table format 802 can include additional rowsand/or columns, not shown in FIG. 8A, which may optionally be viewed byscrolling search screen 800, or by other suitable means. The scrollingmay reveal additional events and/or additional event attributes onsearch screen 800.

In the example shown, each row corresponds to an event. By way ofexample, search screen 800 is shown as displaying events 1, 2, 3, 4, and5, which are each search results of a search result set that can includeany number of additional results that are not shown. A search resultsset may generally include any number of search results. Each columncorresponds to an event attribute, such as an extracted field, metadataabout events (e.g., a default field), or event raw data. Each eventattribute is assigned a respective attribute label, which can beutilized to represent the corresponding attribute in the table format.For example, row 806 a corresponds to event 1 and column 804 acorresponds to an event attribute of event 1 having an attribute labelof _time, comprising a timestamp data item. Other attribute labels shownin FIG. 8A include _raw, corresponding to event raw data, source andhost corresponding to metadata, and bytes, clientip, method, andreferrer, corresponding to extracted fields.

The rows in table format 802 form cells with the columns, and each cellcomprises a data item of an event attribute of a corresponding column.For example, in search screen 800, the event attribute having attributelabel _time has a value of 2013-11-10T18:22:16.000-0800 for event 1,2013-11-10T18:22:15.000-0800 for event 2, and so on. Data items ofevents are shown in FIG. 8A by a textual representation of their value.Although not the case in the table formats shown herein, it is possiblethat the underlying value could vary from the textual representation. Asan example, numeric data items could be textually represented in roundedform. Furthermore, some of the data items could be represented by meansother than a textual representation.

Data items in a column are assigned to an event attribute forming setsof attribute-data item pairs, with each data item corresponding to arespective event for that event attribute. Search screen 800 shows dataitems of multiple events for each event attribute. The displayed eventattributes correspond to the search result set, but other eventattributes corresponding to the search result set could potentially behidden from view or not included in a column.

In the present example, each row in table format 802 corresponds to arespective event and each column corresponds to a respective eventattribute of multiple events, with each cell comprising one or more dataitems of the respective event attribute of the respective event. In somecases, columns of the table format may be reordered in a displayedsearch interface based on a user interaction with the table format. Forexample, a user may select a column to move (e.g., using a mouse) anddrag the selected column to a new position in the table format.

A search query corresponding to the search result set that is utilizedto populate the cells of table format 802 can have its constituentcommands be at least partially represented in search screen 800. Onesuch approach is illustrated by command entry list 808 in search screen800. Exemplary command entry lists will later be described in additionaldetail, and for purposes of the present disclosure, can be employed withother types of interfaces, which do not necessarily include eventsdisplayed in a table format.

Command entry list 808 corresponds to at least a partial representationof the commands of the search query and comprises one or more commandentries. Each command entry is respectively representing one or morecommands of the search query. Furthermore, the list of search commandentries is displayed in a sequence corresponding to a sequence of thesearch commands within the search query. Another approach to at leastpartially representing search commands of the search query in searchscreen 800 comprises utilizing a search bar, in addition to, or insteadof a command entry list. The search bar can accept user input in theform of textual input to a search string that corresponds to the searchquery.

2.2 Exemplary Interactions with a Table Format

In some implementations, a user can interact with one or more events ofa set of events (e.g., a search result set) that are used to populate atable format by interacting with the table format. For example, a usercan interact with table format 802, which is populated with at leastsome data items from events that correspond to the search result set.

In some respects, a user can make a selection of one or more portions ofthe table format. Based on the selection, the system causes for displayone or more options (e.g., a list of options) corresponding to theselected one or more portions. Based on a user selecting one of thedisplayed options, operations corresponding to the displayed option canbe carried out by the system.

In implementations where the table format is part of a search system,the interactions can be made to create and/or modify search queries. Insome approaches, a user makes a selection of one or more portions of thetable format. Based on the selection, the search system causes fordisplay one or more options (e.g., a list of options) corresponding tothe selection. The search system can cause one or more commands to beadded to a search query that corresponds to the set of events used topopulate the table format, based on a user selecting one of the optionsfrom the list of options. The one or more commands that are added to thesearch query can be based on (e.g., generated based on) at least theoption that is selected by the user, and potentially other factors, suchas one or more data items and/or one or more event attributes in theselected one or more portions of the table format.

Examples of selectable portions of a table format include selectablerows, columns, cells, and text, which are described in additional detailbelow. Depending on the implementation employed, any combination ofthese table elements may or may not be selectable. As an example, insome implementations, one or more cells may be selectable withoutnecessarily requiring other portions of the table format to beselectable (i.e., only cells could be selectable). The same is true forother types of table elements, such as columns, and text. Furthermore,in some cases portions of the table format are individually selectable(e.g., individual cells or individual columns could be selected). Inaddition, or instead, portions may be selectable in groups (e.g.,multiple cells or multiple columns could be selected).

FIGS. 8B, 8C, 8D, 8E, and 8F represent search screen 800 afterrespective portions of table format 802 have been selected by a user.FIG. 8B illustrates where a selected portion is an individual cell(e.g., cell 810), although in some implementations a user may selectmultiple cells. FIG. 8C illustrates where a selected portion is anindividual column (e.g., column 804 a). FIG. 8D illustrates wheremultiple columns are selected (e.g., columns 804 b and 804 c). FIG. 8Eillustrates where table format 802 itself is selected. FIG. 8Fillustrates where a portion of a textual representation (text) of one ormore data items (e.g., portion 814 (a text portion) of textualrepresentation 812) is selected. It is noted that, where multiplecolumns are selected by a user, the selected columns need not beadjacent to one another, as shown. Similarly, where multiple cells areselected in the table format, those selected cells need not be adjacentto one another.

Although many approaches exist for selection of portions of a tableformat, in some implementations, a selectable portion(s) may behighlighted or otherwise emphasized when a pointer that is displayed inthe user interface moves over a particular region of the display (e.g.,a region of the table format) that corresponds to the selectableportion(s). This feature is also referred to as highlight with rollover(e.g., detected when a pointer moves over a region). One or morehighlighted selectable portions can then be selected in response toadditional user input, such as a mouse click or touch input to selectthe selectable portions. A shift-click or other method could be utilizedto select additional selectable portions.

By way of example, in search screen 800, each cell is individuallyselectable and the region for each cell is substantially coextensivewith the cell. For example, in FIG. 8B, region 820 a can be used toselect cell 810, which is shown as being emphasized and selected.Furthermore, each column is individually selectable and the region foreach column is coextensive with the column's header, which comprises anattribute label of the event attribute of the column. For example, inFIG. 8C, region 820 b can be used to select column 804 a, which is shownas being emphasized and selected. In implementations where rows areselectable, the region for a row could similarly correspond to the row'sheader (displaying the event number in the present implementation).

Selection of a textual representation, or a portion thereof, couldoptionally be handled, at least partially using highlight with rollover.In search screen 800, a portion of a textual representation may beselected by the user using a click, drag to highlight, and release, asone example. Any of the various characters in text of a textualrepresentation may be highlighted and selected, and in someimplementations, multiple textual representation portions could beselected from multiple cells.

2.3 Exemplary Options

A variety of approaches are available for presenting options that aredisplayed based on and corresponding to the selection of one or moreportions of the table format. In some implementations, options can bepresented as a list of selectable options. Options may appear in sidebar830, or elsewhere. In FIGS. 8B, 8C, 8D, 8E, and 8F, options aredisplayed in option menus. Display locations of option menus can bebased on the one or more portions of the table format that are selectedby the user. For example, option menus can be configured to appearproximate to (e.g., over, or adjacent to) the one or more selectedportions of the table format, as illustrated by option menus 826 a, 826b, 826 c, 826 d, and 826 e.

FIG. 9 shows exemplary option menu 926 in accordance withimplementations of the present disclosure. Option menu 926 comprisesoptions 930 a, 930 b, 930 c, and 930 d, and is utilized to describeexemplary configurations for option menus, such as any of option menus826 a, 826 b, 826 c, 826 d, and 826 e. In the present implementation,each option can correspond to one or more commands that may be includedin a search query. However, in some cases, options need not correspondto one or more commands that may be included in a search query. Instead,the option may be operable to interact with the system in some othermanner. Where, an option corresponds to a command, the command may beprovided to a search query utilizing a format that includes a commandidentifier that identifies the command and one or more command elementsof the command, at least some of which may be optional (e.g., arguments,parameters, values, command options, and the like). In particular, eachcommand could correspond to a pipelined search language command, such asan SPL command, or another type of command compatible with processing ofthe search query.

In various implementations, option menu 926 is a contextual menu. Inthis regard, one or more of the options in option menu 926 can beincluded based on context related to the selection that prompted optionmenu 926. For example, option 930 a may be included in option menu 926in some contexts, but not in others. This may be desirable in that itmay be more intuitive for some options to be invoked in some selectioncontexts, but not in others. Furthermore, one or more options in optionmenu 926 may be a contextual option. In this regard, one or more commendelements for any commands corresponding to an option may be incorporatedinto the commands based on selection context. Furthermore, at least someof the label of the option in option menu 926 can be based on selectioncontext.

In some respects, context can be based, at least partially on the typeor types of table elements of the table format selected by the user. Invarious implementations, the presented option menu may include differentoptions depending on whether the selected portion of the table format isa column, a cell, a portion of a textual representation of a data item,or the table itself. For example, option 930 c may be included in optionmenu 826 b for a selection of column 804 a, but not in option menu 826 afor a selection of cell 810. This may be desirable in that it may bemore intuitive for some options to be invoked through selecting a columnas opposed to a cell, or vice versa. As an example, the option maycorrespond to a command that operates on cells of a column, or thecolumn itself, and therefore might be more intuitive when included as anoption for a selected column. As such, the option might be contextuallyincluded based on selection of a column, but not based on selection of acell.

Context can further be based on a number of selected portions of thetable format, or a number of selected portions of the table format ofone or more particular types of table elements. For example, option 930d may be included in option menu 826 c of FIG. 8D for a selection ofmultiple columns 804 b and 804 c, but not in option menu 826 b of FIG.8C for a selection of only column 804 a. In this way options included inoption menu 826 c may be different than the options included in optionmenu 826 b based on the selected portion comprising two columns in ofFIG. 8D, but only one column in FIG. 8C (e.g., based on the number beinggreater than one column, or based on the number being equal to adesignated number of columns). As another possibility, where selectionportions of a table format comprise a first selected cell and a secondselected cell, an option may be caused to be included in option menu 926based on a determination that the first and second cells are indifferent columns in the table format.

Context can also be based on the event attribute(s) corresponding to theselected portion(s) of the table format. For example, one or moreoptions may be included in option menu 926 based on the user selecting acolumn, cell, or textual representation corresponding to event raw data(e.g., _raw), or based on the user selecting a table portioncorresponding to event timestamps (e.g., _time).

Thus, for example, an option may be included in option menu 926 based ona determination that at least one of one or more data items of at leastone selected one or more cells comprises event raw data, and/or that atleast one of selected one or more columns represent event raw data of aset of events. Furthermore, an option may be included in option menu 926based on a determination that at least one of one or more data items ofat least one selected one or more cells comprises a timestamp of eventraw data, and/or that at least one of selected one or more columnsrepresent data items comprising timestamps of a set of events.

Furthermore, context can be based on one or more data types assigned tothe selected portion(s) of the table format. Examples of data typesinclude numeric data types, categorical data types, and user defineddata types. A numeric data type may correspond to numbers and acategorical data type may correspond to a combination of numbers,letters, and/or other characters. A cell may be identified as comprisinga numeric data type or a categorical data type by the system analyzingthe data item represented in the cell. A column may be identified ascomprising a numeric data type or a categorical data type by the systemanalyzing at least some of the data items represented in the column(e.g., in cells). Such determinations may be made based on the selectionof the table format and prior to selecting an option, or could beperformed prior to the selection of the table format. Furthermore, insome cases, the data type could be selected by a user.

Thus, for example, an option may be included in option menu 926 based ona determination by the system that at least one of one or more dataitems of at least one of selected one or more cells is of a numeric datatype (e.g., based on the overall data type of the cell's correspondingcolumn, or based on the cell's particular data type), and/or that atleast one of selected one or more columns represent data items of anumeric data type. Similarly, an option may be included in option menu926 based on a determination that at least one of one or more data itemsof at least one selected one or more cells is of a categorical datatype, and/or that at least one of selected one or more of the columnsrepresent data items of a categorical data type. As an example, whereeach selected column has a numeric data type, an option may be presentedthat corresponds to one or more commands that apply at least onestatistical functions to the data items of the columns, and/or generatea graph where each axis represents a respective one of the columns. As afurther example, the one or more commands be operable to remove one ormore non-numeric cells from selected one or more cells and/or selectedone or more columns, where at least one non-numeric cell is detected ina selection.

As a further example, context can be based on a source of data items inthe selected portion(s) of the table format. For example, one or moreoptions may be included in or excluded from option menu 926 based on adetermination by the system that at least one of one or more data itemsof at least one of selected one or more cells comprises a statisticalvalue generated by one or more statistical functions performed on valuesof data items of at least some events and/or that at least one ofselected one or more columns represents data items comprisingstatistical values generated by one or more statistical functionsperformed on values of data items of at least some events. As anexample, the system may refrain from offering one or more optionsrelated to extracting new data items from data items that comprisestatistical values. A statistical value may refer to a value generatedfrom an event using one or more statistical functions (e.g., average,sum, mean, median, mode, standard deviation, variance, count, range),such that the value no longer corresponds directly to event raw data. Insome cases, a value may be determined as a statistical value based onidentifying the value as an output of a statistical command in a searchquery. For example, statistical commands may be commands known toproduce one or more statistical values an output.

As another example, one or more options may be included in or excludedfrom option menu 926 based on a determination by the system that atleast one of one or more selected one or more cells is an empty cell,and/or that at least one of selected one or more columns comprises oneor more empty cells. As an example, the system may offer one or moreoptions related to one or more commands that are operable to remove orotherwise perform some operation on empty cells based on a determinationthat a selection comprises at least one empty cell. Furthermore, one ormore options displayed for selections comprising empty cells may excludeone or more options otherwise displayed where the selection does notcomprise empty cells.

In further cases, one or more options may be included in or excludedfrom option menu 926 based on a determination by the system that atleast one of one or more data items of at least one of selected one ormore cells comprises multiple values, and/or that at least one ofselected one or more columns represents one or more cells comprisingmultiple values. A data item that comprises multiple values may comprisean array, matrix, or other representation of multiple values for asingle event attribute of a single event. Each value could be displayedin the same cell and may be displayed in a manner that indicates thevalues as being different values for the same event and event attribute.As an example, where a user selects a portion of a textualrepresentation of a data item in a cell, the system may offer or refrainfrom offering one or more options corresponding to one or more commands,based on whether the data item comprises multiple values.

In option menu 926, options 930 a and 930 b are examples of form-basedoptions, which include at least one form element that can be modified bya user. A form-based option may have at least some defaults entered intothe form. Examples of form elements that may be included in a form-basedoption include one or more of a text box, dropdown list, radio button,checkbox, and the like. Where an option corresponds to one or morecommands, a form element could be employed for selecting and/or enteringone or more command elements for a command and/or a command identifier(so as to select from the one or more commands) Optionally defaultcommand elements and/or command identifiers may be automatically enteredinto the form prior to or after selection of the option. An exemplaryform element could be a dropdown list that comprises a list ofpossibilities for command elements and/or command identifiers. Asanother example, a text box could be used to enter one or more commandelements of a command. The text box could comprise placeholder text thatis descriptive of a command element corresponding to the text box.

A form-based option may be selected by a user using a correspondingapply button, or other suitable means. For example, option 930 a couldbe selected by clicking on apply button 932 after providing input toconfigure the form, or optionally without configuration where the formcomprises one or more default values. An option, such as option 930 c,could be a nested form-based option, where option menu 926 comprises alink that can be selected by the user to open a form of the nestedform-based option. The form may open within or outside of option menu926 (e.g., replace the link with the form, expand the link to displaythe form, or appear outside of the option menu). The form couldsimilarly include an apply button to select the option. Option 930 d isan example of an option that does not include a form. As an example,option 930 d may be selected upon mouse up or mouse down. The optioncould include one or more default command elements. In addition, orinstead one or more of the command elements could be contextuallygenerated, for example, based on the user selection.

As described above, the system can cause one or more commands to beadded to a search query that corresponds to a group of events used topopulate the table format, based on a user selecting one of the optionsfrom the list of options. For example, upon a user selecting an option,the one or more commands could be automatically added to the searchquery. Where the search query employs a pipelined search language, theone or more commands can be added sequentially to the end of the searchquery. In implementations where the search query is displayed to theuser, for example, in a search bar, the one or more commands may beadded to the search bar. In implementations where the search query isrepresented by a command entry list (e.g., command entry list 808), thecommand entry list may be updated to represent the one or more commandsas one or more command entries.

The one or more commands that are added to the search query may be inproper syntax for the search query, complete with command identifiersand any command elements that are needed or desired for execution of thecommands. In some cases, one or more added command elements for acommand are default command elements associated with a selected option.Furthermore, one or more added command elements for a command could beprovided by the form of a form-based option. As another option, thecommand elements could be contextually generated based on the portion(s)of the table format selected by the user.

The one or more commands corresponding to an option can be contextuallybased on an event attribute that corresponds to a selected portion ofthe table format. For example, the event attribute can be used togenerate at least a portion of one or more of the commands for theoption. As an example, the event attribute, and/or one or more dataitems assigned to the event attribute can be incorporated into at leastone command element of one or more commands that correspond to theoption, or used to generate at least one value for the at least onecommand element. A reference to an event attribute (e.g., an attributelabel of the event attribute) or a data item (a value thereof), or datagenerated therefrom, could be included in a form element of an option asa default command element for a command. In addition, or instead, one ormore references (or values, or data generated therefrom) could beincluded as text in a command string added to the search query and usedto invoke a command. The command string can include a command identifieralong with the reference(s), value(s), or data generated therefrom, usedfor a command element. As one example, a command element that is basedon the event attribute may be a command element that instructs thecommand as to which event attribute and/or data item or items assignedto an event attribute to operate on, for example, within events inputinto the command.

Thus, where a user selects a column, one or more commands for an optionmay be based on (e.g., generated using) any event attributescorresponding to the column. For example, at least one command elementmay be generated from the event attribute(s) and/or one or more dataitems that are assigned to the event attribute(s), or values thereof. Asan example, the user could select column 804 a, as in FIG. 8C. A commandelement for a command corresponding to an option in option menu 826 bcould instruct the command to operate on data items of an eventattribute having the attribute label _time, based on column 804 acomprising the event attribute, or could provide data generated from oneor more of the data items as input to the command. Similarly, a usercould select both columns 804 b and 804 c, as in FIG. 8D. A commandelement of a command corresponding to an option in option menu 826 ccould instruct the command to operate on at least some data items of theevent attributes having the attribute labels of source and host, basedon columns 804 b and 804 c respectively comprising those eventattributes, or could provide data generated from one or more of the dataitems as input to the command.

For a cell, the user could select cell 810, as in FIG. 8B, and a commandelement of a command corresponding to an option in option menu 826 acould be generated to instruct the command to operate on the data itemassigned to the event attribute having the attribute label referrer, touse the value of that data item as an input to the command, or togenerate data from the value of the data item as an input to thecommand, each based on cell 810 corresponding to a data item assigned tothe event attribute.

For text, the user could select portion 814 of textual representation812, as in FIG. 8F, and a command element of a command corresponding toan option in option menu 826 e could be generated to instruct thecommand to operate on at least the portion of the text in the data itemassigned to the event attribute having the attribute label “_(J)aw,” touse at least the portion as an input to the command, or to generate databased on the portion as an input to the command (e.g., a keyword thatincludes the portion), all based on portion 814 being in a data itemassigned to the event attribute.

As discussed above, based on the selection of one or more displayedoptions, operations corresponding to a displayed option that is selectedby the user can then be carried out by the system. As one example, whenan option is selected (e.g., in option menu 926), the operations may beautomatically performed. Furthermore, the screen can be updated based onany changes corresponding to the selected options. For example, insearch screen 800, when a user selects an option, the set of eventsutilized to populate table format 802 (e.g., a search results set) maybe automatically updated by the operations associated with the option.As an example, one or more portions of a search query could be executed,as needed to accurately portray events corresponding to the search queryin the table format. Furthermore, the displayed table format 802 may beautomatically updated to reflect changes to the set of events. Moreparticularly, where one or more commands are added to a search query, orthe search query is otherwise modified by an option, table format 802can be automatically updated to correspond to the modified search query.This could result in more or fewer events being included in the tableformat, and/or more or fewer event attributes being included in thetable format, depending on the commands.

Thus, for example, a user may directly interact with the table format tomanipulate a corresponding search query and automatically see theresults of the manipulations reflected in the table format. In doing so,the user need not necessarily directly code the search query, which canrequire extensive knowledge of the underlying search query language.Instead, complicated aspects of coding the search query can be embeddedin the options, and results of the options (and any underlying commands)can quickly be portrayed to the user. For example, the user could selectan option to remove a column comprising an event attribute, and acommand assigned to the option that operates to remove the eventattribute from inputted events can automatically be added to the searchquery. At least the added command could automatically be executed, andthe displayed table format could be updated to no longer include thecolumn, as the event attribute would be excluded from search results.

It should be noted that execution of a search query (or one or moreportions thereof), as described herein can comprise an automaticreformulation of the search query (or one or more portions thereof), soas to more efficiently achieve equivalent search results as the searchquery.

Below, various potential options are described with respect to userselections of a table format. Although options may be described ascorresponding to a single command, similar functionality may be achievedutilized multiple commands. It is further noted that an omission of aparticular option type from a particular option menu is not intended tolimit the option from potentially being included from the option menu.As an example, although extraction type options are only shown in optionmenu 826 e of FIG. 8F, similar options are contemplated as beingincluded for extracting new fields for option menus 826 a, 826 b, 826 c,and 826 d.

In FIG. 8B, option menu 826 a is shown as including two options. Thefirst option corresponds to a command that is operable to filter outeach event input into the command that does not include the value“http://www.buttercupgames.com/productscreen?productid=5F-BV5-G01” for adata item of an event attribute labeled “referrer.” The value andattribute may be provided to the command from the value and eventattribute associated with the user selection. The second option issimilar to the first option, but filters out each event input into thecommand that does include the specified value.

In FIG. 8C, option menu 826 b is shown as including sixteen options,with a filter text box at the top. The filter text box can be used tofilter out events input into a command that do not include any keywordsentered into the text box by a user. Those keywords may be incorporatedinto the command. The first option may be similar to the first option inoption menu 826 a, where the value may be entered by the user.

The second option is associated with a command that is operable toremove events input into the command that contain an identicalcombination of values for an event attribute. The event attribute may beprovided to the command as the event attribute associated with theselected column.

The third option may correspond to a command that is operable to sortthe events input into the command in ascending order of values for anevent attribute, where the event attribute may be provided to thecommand as the event attribute associated with the selected column.

The fourth option is similar to the third option, but sorts the eventsin descending order of the values.

The fifth option corresponds to a command that is operable to change theattribute label of an event attribute for an associated column. Theevent attribute may be provided to the command as the event attributeassociated with the selected column. Furthermore, the new attributelabel for the command may be entered into the text box by a user.

The sixth option corresponds to a command that is operable to remove anevent attribute from events input into the command. The event attributemay be provided to the command as the event attribute associated withthe selected column.

The seventh option corresponds to a command that is operable to removeevents input into the command that have an empty cell, or no value, fora given event attribute of events. The event attribute may be providedto the command as the event attribute associated with the selectedcolumn.

The eight option corresponds to a command that is operable to apply alookup table to changes values for a given event attribute of events.The event attribute may be provided to the command as the eventattribute associated with the selected column.

The ninth option corresponds to a command that is operable to extractall fields discovered within data items for a given event attribute ofevents. Such an option is later described in additional detail. Theevent attribute may be provided to the command as the event attributeassociated with the selected column.

The tenth option corresponds to a command that is operable to split agiven event attribute for events input to the command into one or moreother event attributes (e.g., resulting in additional columns) The eventattribute may be provided to the command as the event attributeassociated with the selected column.

The eleventh option corresponds to a command that is operable toevaluate an expression for each value of a given event attribute forevents input into the command and assign the resulting value to an eventattribute for the event that had its value evaluated. The eventattribute may be provided to the command as the event attributeassociated with the selected column. Furthermore, a user may use thefirst text box to specify an attribute label for the resulting value anduse the second text box to specify the evaluation expression, which areused as command elements in the command. If an event attribute alreadyexists that has the attribute label entered by the user, that eventattribute may optionally be overwritten with the resulting values.

The twelfth option corresponds to a command that is operable to findtransactions based on events input to the command that meet variousconstraints. Events may be grouped into transactions based on the valuesof a given event attribute. The event attribute may be provided to thecommand as the event attribute associated with the selected column.Furthermore, a user may use the first text box to specify a max pausevalue for the command and use the second text box to specify a max spanvalue for the command. The maxspan constraint requires the transaction'sevents to span less than maxspan. The maxpause constraint requires therebe no pause between a transaction's events of greater than maxpause.

The thirteenth option corresponds to a command that is operable todisplay the most common values for a given event attribute in a set ofevents input to the command. The event attribute may be provided to thecommand as the event attribute associated with the selected column.

The fourteenth option corresponds to a command that is similar to thecommand of the thirteenth option, but groups the top values by the eventattribute having the attribute label “_time.”

The fifteenth option corresponds to a command that is operable todisplay the least common values for a given event attribute in a set ofevents input to the command. The event attribute may be provided to thecommand as the event attribute associated with the selected column.

The sixteenth option corresponds to a command that is operable toprovide statistics on values for a given event attribute in a set ofevents input to the command grouped by the event attribute having theattribute label “host.” The given event attribute may be provided to thecommand as the event attribute associated with the selected column.

In FIG. 8D, option menu 826 c is shown as including five options. Thefirst option corresponds to a statistical command that is operable toperform a summation of values of events input to the command for a firstevent attribute grouped by a second event attribute. The first andsecond event attributes may be provided to the command as the eventattributes associated with the selected columns. Such a command mightonly be included in option menu 826 c where at least one of the selectedcolumns is of a numerical data type (e.g., where one is of a numericaldata type and another is of a categorical data type). Furthermore, theevent attribute associated with a selected column having a numericaldata type may be used as the first event attribute and the eventattribute associated with a selected column having a categorical datatype may be used as the second event attribute.

The second option corresponds to a command that is similar to thecommand of the first option, but is operable to perform an averagerather than a summation.

The third option corresponds to a command that is operable to correlatevalues between event attributes of events input to the command to showthe co-occurrence between the values. The command may build acontingency table, comprising a co-occurrence matrix for the values ofthe event attributes. The event attributes may be provided to thecommand as the event attributes associated with selected columns.

The fourth option corresponds to a command that is operable to correlateevent attributes of events input to the command to show theco-occurrence between the event attributes. The event attributes may beprovided to the command as the event attributes associated with selectedcolumns.

The fifth option corresponds to a command that is operable to filter outall event attributes from events input to the command except for givenevent attributes. The given event attributes may be provided to thecommand as the event attributes associated with selected columns.

In FIG. 8E, option menu 826 d is shown as including five options. Thefirst option corresponds to a command that is operable to count thenumber of events input into the command.

The second option corresponds to a command that is operable to count thenumber of events input into the command by the event attribute havingthe attribute label “_time.”

The third option corresponds to a command that is operable to transposeevents input to the command and event attributes of the commands, suchthat each row may become a column.

The fourth option corresponds to a command that is operable to returnthe first N events input to the command where N is a positive integer(e.g., 10).

The fifth option corresponds to a command that is operable to return thelast N events input to the command where N is a positive integer (e.g.,10).

In FIG. 8F, option menu 826 e is shown as including six options. Thefirst option corresponds to a command that is operable to filter outevents input to the command that do not include a given keyword orphrase for a given event attribute. The given event keyword or phrasemay be provided to the command as identified from at least a selectionportion of the textual representation of a data item, and the givenevent attribute can be provided by the event attribute associated withthe data item.

The second option corresponds to a command that is similar to the firstoption, but removes events that do not include the given keyword orphrase for the given event attribute.

The third option corresponds to a command that is similar to the firstoption, but removes events that do not start with the given keyword orphrase for the given event attribute.

The fourth option corresponds to a command that is similar to the firstoption, but removes events that do not end with the given keyword orphrase for the given event attribute.

The fifth option is operable to initiate a field extraction workflow forextracting one or more new fields.

The sixth option corresponds to a command that is operable to extract anew field having a given field label from a given event attribute forevent input to the command. The given field label may be generated fromthe selected portion of the textual representation of a data item, andthe given event attribute can be provided by the event attributeassociated with the data item. Such an extraction may be a suggestedfield extraction, later described in additional detail below.

2.4 Command Entry List

In some respects, the present disclosure relates to a command entrylist, an example of which was briefly discussed with respect to commandentry list 808. Command entry list 808 corresponds to an exemplaryimplementation of a command entry list in a search screen of a searchsystem. However, concepts related to a command entry list are notintended to be specifically tied to such implementations. To thiseffect, command entry lists are discussed in additional detail belowwith respect to FIG. 10 and command entry lists 1008 a and 1008 b.

Command entry lists 1008 a and 1008 b each represent potentialimplementations of command entry lists, in accordance with conceptsdisclosed herein. Command entry lists 1008 a and 1008 b each comprise alist of command entries, which can be displayed in a search interface(such as is command entry list 808), or other interface. As shown,command entry lists 1008 a and 1008 b each comprise command entries 1040a, 1040 b, 1040 c, and 1040 d. Although the command entries are listedin a vertical column (with one command entry per row), other listformats could be employed.

Each command entry in a command entry list may represent one or morecommands of a plurality of commands of a search query. By way ofexample, FIG. 10 shows search query 1044 which may corresponds to eachof command entry lists 1008 a and 1008 b. Command entry 1040 acorresponds to commands 1044 a, command entry 1040 b corresponds tocommand 1044 b, command entry 1040 c corresponds to command 1044 c, andcommand entry 1040 d corresponds to command 1044 d, by way of example.

As shown, the list of command entries of command entry lists 1008 a and1008 b are displayed in a sequence corresponding to the plurality ofcommands of the search query. In particular, the command entries aredisplayed in the list in the same sequence as their correspondingcommands appear in the search query. As the search query utilizes apipelined search language in the present examples, each command entrythat corresponds to a command may be considered a data processing pipeand the sequencing can portray to the user the relationship betweencommands in terms of inputs and outputs for the data processing pipes.It is noted that, the display in the sequence could be visually conveyedusing a variety of possible approaches, such as by depicting acombination of alphanumeric characters proximate to each command entry,and/or the by the positioning of the command entries on screen.Furthermore, in some implementations, the list of command entries of acommand entry list need not be displayed in a sequence corresponding toa plurality of commands of a search query in every implementation of thepresent disclosure.

Command entries 1040 b, 1040 c, and 1040 d each display a representationof their respective underlying commands. In the approach depicted incommand entry list 1008 a, each command is listed as the command wouldappear in search query 1044. Pipes (e.g., “1”) or command separators,are omitted from display, and may be implied from the depiction of thecommand entries. However, in some implementations, command separatorsmight be depicted, or a subset of command separators might be depicted,for example, within a command entry that corresponds to multiplecommands. In the approach depicted in command entry list 1008 b, variousportions of commands are formatted in different manners, to improvelegibility for users. For example, command identifiers are followed bycolons and are aligned to the colons, and might be bolded. Commandelement parameters are formatted in a distinguishable manner using adifferent color of text than other portions, and command modifiers aredistinguishable as unformatted text. Command variables are depicted intext boxes.

In some respects, through interaction with a command entry list, a usercan modify the search query. For example, one to all of the commandentries can correspond to a respective form for modifying its respectiveunderlying commands in the search query. In the approach depicted incommand entry list 1008 a, text of any portion of a command may bemodified. For example, each command entry comprises a text box thatincludes the command A user may modify the command by modifying the textin the text box. The corresponding command in search query 1044 can beupdated accordingly to correspond to the modified command Thus, a usercould change “limit=10” in command entry 1040 d to “limit=20,” andcommand 1044 d may be modified to reflect the changes made using commandentry 1040 d. Such changes may be reflected automatically, or may firstrequire the user to apply the changes.

In the approach depicted in in command entry list 1008 b, portions ofthe commands are represented by a respective form element. Users mayoptionally be restricted from modifying at least some portions of thecommands in the command entries, such as command identifiers.Furthermore, some portions of the commands may have respective dropdownlists, or option menus, associated therewith that can be used to selectfields, aggregation methods, or other command parameters to be used asthe portion of the command. The options presented for a given portion ofa command may be included based on the type of command element suitablefor the position of the given portion in the command. As shown, count,referrer, host, and remove all have corresponding option menus,indicated as rounded rectangular boxes (e.g., 1041 in FIG. 10).

In some cases, a user interaction with the command entry list may breaka dependency of a command element(s) of one or more command entries. Forexample, a user could change “field—_raw” in command entry 1040 b to“field—host” and command 1044 b may be modified to reflect the changemade to command entry 1040 b. In the present example, “_raw” and “host”are used as command elements that instruct the command having commandidentifier “field” as to which event attribute to operate on. Whenexecuted, data items of the event attribute may be removed from eventsinput into the command Thus, due to the modification “_raw” data itemsmay no longer be removed, but “host” data items may be removed instead.However, as shown, command entry 1040 c has a command elementinstructing the command to operate on an event attribute referenced by“host.” In some implementations, in response to the user adding “host”to command entry 1040 b, the system may optionally detect that themodification to command entry 1040 b breaks a dependency of the commandelement (e.g., event attribute) referenced in command entry 1040 c andautomatically throw that command entry into an error state. In the errorstate, the commands associated with the command entry may be excludedfrom execution in the search query and a visual indication may bedisplayed to the user that the command entry is in the error state.

Although some modifications may throw one or more command entries intoan error state, in some cases, a broken dependency caused by a userinteraction may be automatically identified and corrected in the searchquery. For example, assume that a first command entry represents acommand in a search query that is executable to extract a new field froman event attribute and names that field “referrer.” The command mayinclude a command element “referrer,” which instructs the command tolabel the new field “referrer.” Also assume that a second command entryrepresents a subsequent command in the search query that is executableto operate on a field labeled “referrer.” In accordance withimplementations of the present disclosure, the system may automaticallydetect a dependency between the commands of the command entries when auser renames “referrer” through interaction with the first commandentry. Based on detecting the dependency, the system may automaticallyrename the command element (e.g., event attribute) in the command(s)represented by the second command entry (and potentially any otherdependent commands in the search query) to correspond to or match therenamed command element represented by the first command entry. Thus,where a user interaction comprises a user renaming a command element inone or more commands using a form element in a designated command entrythat represents the command element, and one or more other commandelements in one or more other commands can be automatically renamed inthe search query so as to correspond to the renamed command element. Inthis way, error states can be avoided for command entries representingcommands that include command elements that depend on (e.g., reference)command elements of other commands.

The approach depicted in command entry list 1008 b may be similar tocommand entry list 1008 a, but with only designated portions of acommand being directly modifiable by a user, while at least someportions are not directly modifiable by the user. By way of example,command identifiers are not directly modifiable through the depictedcommand entries. However, as indicated using underlining, commandvariables are directly modifiable as text boxes. Command elementparameters may optionally be modifiable using respective dropdown lists,as one example.

As shown in FIG. 10, each command entry has one or more correspondingselectable options to modify the command entry list with respect to thecommand entry. The selectable options are accessible through respectiveform elements. As an example, in each of command entry lists 1008 a and1008 b, command entries 1040 b, 1040 c, and 1040 d each compriserespective options 1042 a, 1042 b, and 1042 c. The options fora commandentry are selectable for the command entry by way of one or morerespective form elements, which are visually and operably associatedwith the command entry. In command entry list 1008 a, two form elementsare employed for each of command entries 1040 b, 1040 c, and 1040 d. Inparticular, a button (e.g., x-button), and dropdown list are includedfor each command entry. The option corresponding to the button (which inother implementations could be selectable in some other manner) incommand entry list 1008 a is operable to delete at least thecorresponding command entry from command entry list 1008 a Similarfunctionality may be incorporated into the options, as shown in commandentry list 1008 b.

Deleting the corresponding command entry may automatically shiftpositions of the subsequent command entries in the command entry list tofill the gap left by deleting the command entry, while otherwiseretaining sequencing of the command entries in the command entry list.Furthermore, deleting the command entry may delete (e.g., automatically)the one or more commands that correspond to the command entry fromsearch query 1044 (and optionally one or more command separators).Similar to the command entries, any subsequent commands in the searchquery may have their sequencing retained, with a command thatimmediately preceded any deleted command(s) being configured to receivethe input that would otherwise had been provided to the deletedcommand(s) Thus, through selection of the option, both the search queryand the command entry list may be updated.

As other potential options, the options for a command entry can beselectable to reorder the command entry within the command entry list.One exemplary reorder option is a shift up option, which is operable toswap positions of the command entry associated with the option with thecommand entry immediately preceding the command entry in the list.Swapping a position of the command entry may automatically swap positionof the one or more commands represented by the associated command entrywith the one or more commands represented by the immediately precedingcommand entry in search query 1044. Another example of a reorder optionis a shift down option, which is similar to the shift up option with theimmediately preceding command entry being substituted with theimmediately following command entry for swapping. It will be appreciatedthat other reordering options are possible, and each may comprisemodifying the sequencing of command entries in the command entry listand modifying the search query to reflect corresponding changes.Furthermore, in some cases, a user may interact with the command entrylist by selecting a command entry (e.g., using a mouse), and draggingthe command entry to a different position in the command entry list,thereby reordering the command entry list.

In addition to command entries, a command entry list may include a blankentry, such as blank entry 1050. Blank entry 1050 is operable to add newcommand entries to the command entry list. In command entry list 1008 a,blank entry 1050 comprises a form that is operable by a user to input acommand into blank entry 1050. In the example shown, the form comprisesa text box for inputting the command as text, although other formelements could be employed. As shown, the text box is created withplaceholder text that reads “Add new pipe . . . ” and may be utilized toapply blank entry 1050, with a command entered therein by a user, tocommand entry list 1008 a as a new command entry and/or to include thecommand entered therein in search query 1044 (e.g., add the new commandentry to the end of the list of command entries and/or add the newcommand(s) to the end of the search query). One or more associatedoptions 1052 may be incorporated into one or more form elements inaddition to or instead of the text box. Options 1052 may provide a menuof commands that a user can add to the command entry list to create anew command entry by selecting its associated option.

It should be noted that changes made to the command entry list using aform element may be applied to the search query as part of operationsassociated with the form element or the form containing the form element(e.g., automatically). However in some implementations, the user maymake multiple changes to command entries or sequencing of the commandentries in the command entry list, and the changes are individually orcollectively applied to the search query by the user, for example, usingone or more apply buttons, and/or save features.

In FIG. 10, command entry 1040 a is an input command entry. An inputcommand entry can optionally be included in a command entry list andcorresponds to an input query (e.g., 1044 a), or pipeline, which servesas an input to the subsequent commands of the search query. Therefore,an input command entry may be the first command entry in a command entrylist, if present. The input command entry may be permitted to representmore than one command, while command entries 1040 b, 1040 c, and 1040 dmay optionally be restricted to representing single commands One or morecommands corresponding to an input command entry may optionally behidden from display in the interface and/or the command entry list(although they may optionally be capable of being revealed in somemanner). Furthermore, the interface and/or the command entry list mayrestrict the user from directly modifying the commands represented bythe input command entry.

In some implementations, an input command entry corresponds to a savedinput query or pipeline that is selected by the user as an input. “WebLogs” in FIG. 10 is an example of a label corresponding to a saved inputquery or pipeline that can be displayed with the command entry list. Theuser can optionally be permitted to selectively replace the saved inputquery with a different input search query (e.g., another saved query)through the interface comprising the command entry list. As an example,form element 1056 could comprise an option operable to initiate such aselection. It should be appreciated that search query 1044 can beupdated accordingly to replace commands of a previous input search query(e.g., commands 1044 a) with a newly selected input search query orpipeline.

From the foregoing, it should be appreciated that one or more commandentry lists can be employed to create and/or modify a search query. Itis further noted that, command entry lists can be incorporated intointerfaces that allow search queries to be created and/or modified usingadditional means. Command entry list 808 in FIG. 8A is one such examplewhere, as described above, a user may modify or create the search query,for example, through interactions with table format 802, or possiblyusing a search bar. Thus, the search query described with respect tosearch screen 800 could correspond to search query 1044 of FIG. 10.

In these cases, where the search query is modified through tableinteractions, the search bar, or other means, the changes may bepropagated to the command entry list (automatically or otherwise). Toillustrate the foregoing, when a user selects an option, such as one ofthe options in option menu 926, one or more commands corresponding tothe option can be automatically added to the end of command entry list808 as a command entry. In some cases, where the option is a form-basedoption, the form corresponding to the option might be reproduced in thecommand entry, as completed by the user. In particular, each of commandentries 1040 b, 1040 c, and 1040 d in command entry list 1008 b couldcorrespond to a form of a respective option previously selected by auser. The form elements utilized for creation and/or modification of thecommand in the form of the option can be reproduced, as shown.

2.5 Exemplary Pipeline Selection Interface

In further respects, implementations of the present disclosure relate todata processing pipelines (or simply pipelines) that are defined by oneor more search queries. A data processing pipeline can correspond to aset of sequenced commands configured such that inputs to the pipelineare operated on by a first command in the sequence, and each subsequentcommand operates on results produced by the preceding command in thesequence, until a final command in the sequence provides one or moreoutputs to the pipeline. The sequence of the commands can be defined bya search query, such as by using a pipelined search language like SPL.

In various implementations, a search query can be created that defines adata processing pipeline that extends another data processing pipeline,which itself may be defined by a search query. In some cases, to createsuch a search query, one or more data processing pipelines can beselected as a basis for the search query (e.g., as an input search queryor pipeline, as described above). In doing so, the full search queryneed not be created (e.g., by a user), instead, only an additional querymay be defined that corresponds to an extended portion of the selecteddata processing pipeline(s).

In some implementations, a user can select the one or more saved dataprocessing pipelines as a basis for the search query using a selectioninterface. The selection interface could optionally be displayed withanother interface, such as a search interface, in the same screen as theother interface, or in a different screen. An example of a suitableselection interface is described with respect to selection interface1100 of FIG. 11.

As shown, selection interface 1100 comprises a plurality of pipelineentries, such as pipeline entries 1102 a and 1102 b, which are listed inselection interface 1100. Each pipeline entry represents a savedpipeline (e.g., a persistently stored pipeline) and displays a pipelinelabel assigned to the saved pipeline. For example, pipeline entry 1102 ahas pipeline label 1104 a that reads “All Data,” and represents arespective saved pipeline that could optionally serve as a basis for allother pipelines. As another example, pipeline entry 1102 b has pipelinelabel 1104 b that reads “Web Logs,” and represents a respective savedpipeline. Each saved pipeline can represent a saved search query thatdefines the saved pipeline. As an example, the saved pipelinecorresponding to pipeline entry 1102 b can represent a saved searchquery comprising commands 1044 a of FIG. 10, by way of example.

Each pipeline entry is selectable to load the saved pipeline thatcorresponds to the pipeline entry. For example, each pipeline entry canhave one or more links that are selectable to load the correspondingpipeline. In the example shown, two links are included for each pipelineentry, which are “edit pipeline” links and “use as input” links. Asshown, pipeline entry 1102 a comprises edit pipeline link 1108 a and useas input link 1106 a and pipeline entry 1102 b comprises edit pipelinelink 1108 b and use as input link 1106 b.

To this effect, in some implementations, a user may assign one or moretags to any pipeline entry. As an example, pipeline entry 1102 a has anassigned “data set” tag. Other tags for other shown include “mine,”“accelerated,” and “lookup.” A user can filter out pipeline entries fromselection interface 1100 based on their assigned tags. For example,filter form 1112 can be used to select which tag(s) pipeline entriesshould have to be included in selection interface 1100. As anotherpossible feature, each pipeline entry has a down chevron, which isselectable to show configuration settings for that pipeline entry. Asshown, the pipeline entry with a pipeline label that read “Errors in thelast 24 hours” has been selected, and thereby expanded to show itsconfiguration settings. Some of the configuration settings may bechanged through selection interface 1100, such as those shown asincluding “Edit” links that may be selected to edit a correspondingsetting.

In response to a user selecting an edit pipeline link, the system maycause the corresponding pipeline of the pipeline entry to be loaded forediting. By selecting edit pipeline link 1108 b, for example, a savedpipeline corresponding to commands 1044 a in FIG. 10 could be loadedinto an interface where the user may modify the pipeline. Loading asaved pipeline may comprise loading at least some of a saved searchquery corresponding to the saved pipeline. Thus, commands 1044 a couldbe loaded into an interface. A user may modify at least a portion of thesaved search query by adding, deleting, and/or modifying one or morecommands of the pipeline using the interface.

The saved pipeline may be loaded into a search interface, which maycorrespond to search screen 800, as one example. Thus, the savedpipeline could be used as the search query described above that can bemodified using the search interface. In addition, or instead, the savedpipeline could be loaded as the search query described above as beingmodified using a command entry list by populating the command entry listwith command entries representing the saved pipeline. However, the savedpipeline may be loaded into other interfaces, which may still implementan interactive table format or command entry list, as described above,or another type of interface. The interface may or may not displayevents that correspond to the search query. In implementations where theevents are displayed, upon loading the saved pipeline, events may beloaded and/or displayed that correspond to the saved pipeline (e.g., anoutput data set of the saved pipeline).

The events could be loaded by optionally executing the saved pipeline.However, in some implementations, the events could be loaded from anexisting data set corresponding to the saved data pipeline. In somecases, the data set may be saved (e.g., persistently) in associationwith the saved data pipeline and may correspond to an output data set ofthe saved data pipeline. A saved data set could be saved as a table, orin another format. By loading the saved data set, the saved pipelineneed not be executed to load the events, thereby saving systemresources. Furthermore, where one or more commands are added to thesaved pipeline, the one or more commands may use the data set as aninput data set for further processing, rather than executing the entirepipeline.

Any modifications that may have been made to the loaded pipeline may besaved, for example, to the saved pipeline, such that the saved pipelineis updated to correspond to the modified search query (the save processmay be initiated by the user, for example, by selecting a save option inthe interface). Furthermore, an updated output data set may be saved inassociation with the updated saved pipeline, which may replace apreviously saved data set. The updated saved pipeline and/or updatedsaved data set may then be accessed using the pipeline entry in theselection interface, for example, through edit pipeline link 1108 b oruse as input link 1106 b.

In response to a user selecting a use as input link, the system maycause the corresponding pipeline of the pipeline entry to be loaded asan input pipeline for an existing search query, or as a basis for a newpipeline that may be created based on the input pipeline. By selectinguse as input link 1106 b, for example, a saved pipeline corresponding tocommands 1044 a in FIG. 10 could be loaded into an interface forcreating and/or modifying a search query that builds on, or extends, thesaved pipeline. Loading a saved pipeline may comprise loading at leastsome of a saved search query corresponding to the saved pipeline. Thus,commands 1044 a could be loaded into the interface. A user may possiblydelete and/or modify the loaded one or more commands using theinterface. However, in some cases, the one or more commands may behidden from the user and/or the interface may preclude the one or morecommands from being modified (e.g., as a default that may be overriddenby the user). Furthermore, the user may be precluded from modifying theunderlying saved pipeline using the interface (at least directly).

The saved pipeline may be loaded into a search interface, which maycorrespond to search screen 800, as one example. Thus, the savedpipeline could be used at least as a basis for the search querydescribed above that can be modified using the search interface. Inaddition, or instead, the saved pipeline could be loaded as at least abasis of the search query described above that may be modified using acommand entry list by populating the command entry list with one or morecommand entries representing the saved pipeline. However, the savedpipeline may be loaded into other interfaces, which may still implementan interactive table format or command entry lists, as described above,or another type of interface. In implementations where the events aredisplayed, upon loading the saved pipeline, events may be loaded and/ordisplayed that correspond to the saved pipeline (e.g., an output dataset of the saved pipeline).

The events could be loaded by optionally executing the saved pipeline.However, in some implementations, the events could be loaded from anexisting data set corresponding to the saved data pipeline. In somecases, the data set may be saved (e.g., persistently) in associationwith the saved data pipeline and may correspond to an output data set ofthe saved data pipeline. By loading the saved data set, the savedpipeline need not be executed, thereby saving system resources.Furthermore, where one or more commands are added to the saved pipeline,the one or more commands may use the data set as an input for furtherprocessing, rather than executing the entire pipeline.

In various implementations, one or more commands can be added to and/oredited in a search query that builds off of, or extends, the loadedsaved pipeline (e.g., commands 1044 a). The search query may correspondto the saved pipeline (e.g., commands 1044 a), with an additional searchquery that builds off of the saved pipeline. One such example is searchquery 1044 in FIG. 10, with the additional search query comprisingcommands 1044 b, 1044 c, and 1044 d that might have been added usingcommand entry list 1008 a or 1008 b. As another example, the additionalsearch query may have been added using a combination of command entrylist 808 and interactions with table format 802, where the interfacecorresponds to search screen 800. Where updated events are needed fordisplay, the updated events may be generated based on the output dataset of the loaded saved pipeline, for example, by executing theadditional search query using the output data set as an input, therebysaving system resources. Furthermore, by preventing the user frommodifying the loaded pipeline in the interface, it may be ensured thatthe output data set can be used regularly for this purpose.

The constructed search query may be saved, for example, as a new savedpipeline that corresponds to the search query (the save process may beinitiated by the user, for example, by selecting a save option in theinterface). Furthermore, an output data set may be saved in associationwith the new saved pipeline that corresponds to an output of thepipeline. Additionally, the constructed search query can be saved inassociation with the saved pipeline and a new pipeline entry. The savedpipeline may then be accessed, for example, in selection interface 1100using an associated use for input pipeline link and an associated editpipeline link in the pipeline entry, similar to web logs.

2.6 Dependent Pipelines

In some respects, the present disclosure relates to creating adependency between a first search query and a second search query, wherethe first search query defines a first data processing pipeline and thesecond search query defines a second data processing pipeline thatextends the first data processing pipeline. The system can detect amodification to the first data processing pipeline defined by the firstsearch query, and based on the modification of the first data processingpipeline being detected, enforce the dependency, such that the seconddata processing pipeline is modified to extend the modified first dataprocessing pipeline.

In the context of the previous example, a dependency may be createdbetween the saved pipeline (input pipeline) and the new pipeline beingmodified in the interface, where the first search query is the querycorresponding to the saved pipeline and the search query being createdcorresponds to the second search query. In some cases, the modificationthat is detected could be performed while a user is modifying the firstsearch query using an edit pipeline link, as one example. For example,the user could be modifying the first search query and creating the newpipeline concurrently (e.g., in different tabs). By enforcing thedependency, the new pipeline is modified to still extend the modifiedfirst data processing pipeline (e.g., automatically). As an example, thechanges made to the first search query may be incorporated into the basepipeline being used to construct the new pipeline. In some cases, anon-persisted data set that corresponds to an output of the modifiedfirst search query (e.g., generated while modifying the first searchquery) could be used as an input to the additional search query, suchthat only the additional search query needs to be executed. As a furtherexample, such a dependency may only be detected and/or enforced based onthe modified first search query being saved by the user to update thesaved pipeline (e.g., persistently).

As another example, in the context of the previous example, a dependencymay be created between the saved pipeline and the newly saved pipeline(e.g. associated with the new pipeline entry), where the first searchquery is a query corresponding to the saved pipeline that served as abasis for the newly saved pipeline, and the second search query is aquery corresponding to the extended saved pipeline. The modificationthat is detected could be performed using an edit pipeline link, as oneexample. Furthermore, the modification could correspond to an update tothe saved pipeline that serves as a basis for the dependent pipeline.The dependency can be enforced so as to ensure that when a user selectsthe dependent pipeline for editing, or as an input pipeline, thepipeline that is loaded extends the updated saved pipeline.

Dependencies can be created and/or saved at any suitable time, such asbased on a user selecting to save a pipeline. As another option, thedependency may be created and/or saved in response to the user selectinga saved data processing pipeline as a basis for the new pipeline (e.g.,prior to any saving of the new pipeline). Furthermore, dependencies canbe enforced at any suitable time. As an example, dependencies and beenforced at load time (e.g., using selection interface 1100), at savetime (e.g., a user selected save process), or during query modification(e.g., using search screen 800).

It is noted that multiple pipelines may be created that extend that samebase pipeline. Thus, dependencies may exist for each of these pipelines,such that changes to the base pipeline are propagated to the dependentpipelines. In this way, a user may only need to modify the basepipeline, instead of having to individually modify the other pipelines.Furthermore, each dependent pipeline may optionally apply its ownadditional processing to the same output data set produced by the basepipeline, without necessarily having to execute the base pipeline eachtime, as one example. Such features may be especially beneficial wherethe pipelines are used to apply a late-binding schema.

It is further noted that pipelines can be created that extend apipeline, which itself extends a base pipeline. In such a case, apipeline may have multiple dependencies, thereby creating a chain ofsaved pipelines. It is further noted that one pipeline can extend morethan one base pipeline. In such a case, the pipeline may also havemultiple dependencies. For example, the base pipelines could correspondto a combination of the base pipelines that may act to join, transact,or otherwise mix the pipelines in processing.

2.7 Extracting Field Label-Value Pairs

In some respects, the present disclosure relates to various approachesfor extracting field label-value pairs from data items of events, suchas event raw data, extracted fields, metadata, or other data items thatmay be assigned to one or more events. These approaches are useful incombination with some implementations described herein, such as varioussearch interfaces. For example, various aspects of these approaches maybe incorporated into at least one of the options that may be presentedbased on a user selecting a portion of a table format, which may be oneof the options in option menu 926 (e.g., as one or more commands).However, it is noted that these approaches are more generally useful inthe context of analyzing and/or interacting with events, which mayoptionally be facilitated by a graphical interface for displaying theevents.

Extracting a field label-value pair from an event can generally refer toa process whereby a field label and a value associated with the fieldlabel are identified from the content of an event, such as event rawdata of the event, or another event attribute, as a field label-valuepair. Extractions of field label-value pairs can be implementedutilizing extraction rules that are applied to data in the events toextract values for specific fields as data items for the fields. In thecontext of extracting field label-value pairs, an extraction rule for afield can include one or more instructions that specify how to extract avalue for the field from event data and further how to identify and/orextract a field label for the value from the event data. In some cases,an extraction rule comprises one or more regular expressions to provideinstructions for identifying a field label and/or value. The field labelmay optionally be assigned to the field.

An extraction rule for extracting field label-value pairs can generallyinclude any type of instruction(s) for identifying and extracting valuesand for identifying and/or extracting a field label corresponding to anyextracted values, from data in events. The field label may optionally begenerated from the data in the events. In contrast, other extractionrules may only provide instructions for identifying and extractingvalues from data in events. A field label may be manually entered for afield associated with the values.

An example of an extraction rule for extracting field label-value pairsis a rule that identifies a field label for a field based on text on theleft hand side of an equal sign (“=”), and identifies a value for a newdata item or value associated with the field label based on text on theright hand side of the equal sign within a value of a data item. Theequal sign can be used to demarcate text representing a field label andtext representing a value associated with that field label. Theidentified text on each side of the equal sign could further bedemarcated by a space character (“ ”). It should be noted that otherdemarcating character(s) could be employed in addition to, or instead ofequal signs and space characters to define text representing a fieldlabel and/or value associated with the field label, such as one or morecolons, back slashes, ampersands, quotation marks, and the like.Furthermore, rules that identify demarcating characters can vary incomplexity, such as by considering text representing a value as beingdemarcated by characters that are not a number or word character. A wordcharacters may include a to z, A to Z, or underscore, as an example.Furthermore, text representing a field label could be demarcated in adifferent manner than text representing the value, for at least oneboundary thereof. Additionally, processing, such as decoding, couldoptionally be applied to the text portions to generate the field labeland/or value.

Using such an extraction rule that identifies field labels and valuesusing one or more demarcating characters, a system can identify withintext in a data item that reads “itemid=EST-14,” “itemid” as a fieldlabel, and “EST-14” as a value for a data item associated with the fieldlabel for a field label-data item pair. Such as in the raw event data ofevent 1 in FIG. 8A. Any values that are extracted from events using anextraction rule may be assigned to a new or existing field of an eventas data items, for example, to define a late-binding schema for events.Thus, with reference to FIG. 8A, using the extraction rule, a new eventattribute (an extracted field) may be created and assigned the extractedfield label “itemid” for each event, along with data items correspondingto the extracted value associated with the field label for that event.As an example, the extraction rule may generate itemid data items havingthe values of “EST-14,” “EST-15,” “EST-18” respectively for events 1, 3,and 4 in FIG. 8A. As events 2 and 5 do not include text in the format of“itemid=,” they could optionally be assigned a data item having a blank,or default value, or no value (e.g., empty) Similar extraction rules canbe applied to each search result. The aforementioned example utilizesevents that are part of search results for convenience only. It is notedthat concepts related to extracting field label-value pairs are moregenerally applicable to any set of events.

2.8 Distinguishing Between Extracted Field Label-Value Pairs andExisting Field-Data Item Pairs

In accordance with some implementations of the present disclosure, anextraction rule for extracting field label-value pairs from events canbe used to extract a field label that corresponds to a field labelassigned to another previously extracted field of the events. Thepreviously extracted field may have been extracted using any suitablemeans, such as an extraction rule that only identifies and extractsvalues for a field, or an extraction rule that extracts fieldlabel-value pairs for a field. The values of the extracted fieldlabel-value pairs may be assigned to data items of another field of theevents (e.g., a new field), in addition to or instead of assigning thevalues to data items of the existing field of the events.

Thus, the newly extracted values can be distinguished from thepreviously extracted values. This approach can be useful in manycontexts, such as to distinguish between values that were extracted fromevents using different extraction rules. As another example, thisapproach can be used to distinguish between values that were extractedfrom one event attribute of events (e.g., event raw data), and valuesthat were extracted from another event attribute of the event (e.g., anextracted field).

Referring to FIG. 8G, exemplary events are shown in table format 802including fields extracted from the events. Assume that field 850 havingfield label 852 “itemid” was extracted from event attribute 854 havingattribute label 856 “_raw” (e.g., event raw data) using an extractionrule which may include a regular expression and may extract valuesassociated with instance 872 of “itemid=”. An extraction rule forextracting field label-value pairs from field 858 having field label 860“referrer” can be used to extract field label 862 that corresponds tofield label 852 “itemid” of previously extracted field 850 of theevents. In accordance with implementations of the present disclosure,the system may cause values 864 of the extracted field label-value pairsto be assigned as data items of another field of the events (e.g., a newfield 866 having field label 868 “referrer itemid” in FIG. 8G), inaddition to, or instead of assigning values 864 to data items ofexisting field 852 of the events (e.g., automatically).

Where a new field is created, in some cases, the new field could beassigned the field label that was associated with the values (e.g.,field label 862). However, in the example shown, new field 866 isassigned another field label 868, such as a modified version of fieldlabel 862. As examples, the assigned field label could be the identifiedfield label prepended, appended, or otherwise supplemented with text,such as user specified text. Instead of user specified text, the textcould be automatically generated. In the example shown, the text is fromattribute or field label 852 that is assigned to event attribute orfield 858 from which values 864 were extracted (i.e., “referrer”). Thus,field label 868 that is assigned to new field 866 identifies the sourceof values 864 for that field.

In accordance with some implementations, the extraction of fieldlabel-value pairs can be invoked using a command A command identifierfor the command could be “autoextract,” as a specific example. Thecommand could optionally allow the user to specify how to supplement theidentified field label, such as the text for supplementing theidentified field label, for example, using prepend (if any). The commandcould also optionally specify which attribute(s) of the events should beused to identify the field label-value pairs. Furthermore, the commandcould also optionally specify which attribute label(s) of the eventsshould be identified as field labels for the field label-value pairs inthe events. In some cases, the command may identify all fieldlabel-value pairs that exist in the events, or only in specifiedattributes thereof. Where an attribute label is specified, the commandmay optionally only identify field label-value pairs that include theattribute label as a field label. Additionally, the command couldoptionally specify whether to decode the field label and/or value duringextraction. Where decoding is specified, the command may further specifywhich decoding libraries should be utilized for the decoding.

An exemplary command in accordance with the forgoing could be invokedwith “autoextract attribute=‘itemid’ prepend=‘new_’ decode=‘url’.” Inthis example, “autoextract” is a command identifier,“attribute=‘itemid’” instructs the command to extract from the eventattribute having an attribute label of “itemid,” and “prepend=‘new_’”instructs the command to prepend any field labels identified in theitemid attribute with “new_.” Furthermore, decode=‘url’ instructs thecommand to use URL decoding (e.g., by referring to a URL decodinglibrary) on any extracted values.

In executing this exemplary autoextract command, the system may searchthrough the data items associated with the itemid attribute for eventsprovided as input to the command. The autoextract command can employextraction rules to extract sets of field label value pairs for eachfield label and associated values identified in the data items. In thiscase, the field labels may not only be identified in the data items, butalso generated, or extracted from the data items. For example, thecommand may search for text in data items having the format “A=B,” whereeach unique A is made a field label for extraction and each B is a valueassociated with the field label, as one example. Using such a command,at least one field label automatically discovered and extracted from thedata items may match or otherwise correspond to a field label of analready existing extracted field. Despite this, the values associatedwith discovered field labels are assigned to data items of new fields,thereby distinguishing the values from the values of already existingfields. Based on the prepend, the new fields generated by theautoextract command will each be assigned field labels comprising thediscovered field labels prepended with “new_.” Thus, a user may easilyidentify the fields in a graphical interface. The autoextract commandmay optionally correspond to the option labeled “Auto-extract” in FIG.8C.

It will be appreciated that many variations of the forgoing arepossible. In the foregoing example, each field label that is discoveredis automatically prepended with text regardless of whether thediscovered field labels correspond to a field label of an alreadyexisting field. In accordance with some implementations of the presentdisclosure, the system can identify (e.g., automatically) where anextraction rule for extracting field label-value pairs from eventsextracts a field label that corresponds to a field label assigned toanother previously extracted (and assigned) field of the events.

Doing so can provide various benefits. In the autoextract command above,for example, the system could optionally treat discovered field labelsthat do not correspond to a field label of an existing field differentfrom those that do (also referred to as “duplicate field labels” forconvenience). As an example, the system may automatically perform someaction, such as presenting one or more options to the user based on theidentification. As another example, the system could apply the prependedtext (or other modified version of a field label) only to duplicatefield labels (e.g., as part of the command), which may be performedautomatically, or could be one of the above options selectable by theuser. However, it is noted that benefits associated with identifyingduplicate field labels in field label-value pair extraction are notlimited to the autoextract command, and are more generally applicable tofield label-value pair extraction.

2.9 Extracting Field Label-Value Pairs from Extracted Fields

In further respects, the preset disclosure relates to extracting fieldlabel-value pairs from data items of a field (e.g., assigned to one ormore events) that were themselves extracted from an event attribute. Forexample, the data items may have been extracted from event raw data, oranother extracted field. Furthermore, the data items may have beenextracted using any suitable extraction rule. Examples of suitableextraction rules include an extraction rule for extracting fieldlabel-value pairs, and an extraction rule that only providesinstruction(s) for identifying and extracting values from data for afield.

Such an approach provides many potential benefits, an example of whichis described with respect to the events of FIG. 8G, by way of exampleonly. As can be seen in FIG. 8G, some of the event raw data shown inevent attribute 854 of events 1, 3, and 4 includes two instances of“itemid=,” instance 870 embedded in a (uniform resource identifier) URIand instance 872. A user may wish to create a field based on instance870 only, as each instance may have a different meaning. Instead of onecomplicated extraction rule, two simpler extraction rules could beemployed to extract the desired data. First, a regular expression basedextraction rule could be used to extract the URI to new field 858 havingfield label 860 “referrer,” as shown in FIG. 8G. Subsequently, fieldlabel-value pair extraction can be performed on extracted field 858, soas to extract values 864 associated with field label 862 “itemid” in theURI, as opposed to anywhere in the raw event data. FIG. 8G showsextracted field 866 having field label 868 “referrer itemid,” which canbe a new field extracted from extracted field 858 having field label 860“referrer.”

As another example, the subsequent extraction could be part of a commandto automatically extract any field label-value pairs it discovers indata items of a specified attribute for events, such as in some of theexemplary autoextract commands described above (e.g., using“Auto-Extract” in FIG. 8C). Applying such as command to the event rawdata of event attribute 854 of events 1, 3, and 4 may extract fieldlabel-value pairs for each field label discovered therein, which mayinclude, itemid, JSESSIONID, categoryid, productid, and action, and alsomight automatically assign those field label-value pairs to fields.However, the user may only be interested in itemid. In accordance withimplementations of the present disclosure, the command could be appliedto an extracted field (e.g., field 858), such that the field labels(e.g., field label 862) that are extracted from the extracted fieldcomprise a subset of the field labels that would have been extracted hadthe same command(s) been applied to event attribute 854 (or anotherevent attribute such as a field). Where the events are displayed in atable format, such as table format 802, such a command(s) could beassociated with an option in an option menu where the user selected acolumn corresponding to the extracted field (e.g., field 858). Amongstother possible benefits, this approach can simplify the user experiencewhile avoiding unwanted or unnecessary extractions.

3.0 Suggested Field Label-Value Pair Extractions

In some respects, the present disclosure relates to suggesting fieldlabel-value pair extractions. One or more field label-value pairextractions can be suggested to a user, and may be suggested based onanalyzing one or more portions of one or more events of a set of events.In some cases, the one or more portions that are analyzed are selectedby a user, such as in a user interface displaying at least some of theset of events. One such suitable interface may display a table format,such as table format 802. Furthermore, the selected one or more portionscould correspond to selected portions of the table format, which couldbe accomplished in a similar manner as described above, amongst otherpossibilities.

In some implementations, the system receives data indicating theselection of one or more portions of data items of a set of events in agraphical interface displaying one or more events of the set of events.Based on the selection, the system automatically detects at least onefield label-value pair at least partially within the selected one ormore portions of the data items. Each detected field label-value paircan include a value and an associated field label. For example, thesystem may automatically determine an extraction rule capable ofextracting a field label-value pair at least partially within at leastselected one or more value, thereby detecting the field label-valuepair.

If a user selects a cell in the table interface, which could correspondto the selection of cell 810 shown in FIG. 8B, the system could attemptto detect at least one field label-value pair that is at least partiallywithin the data item corresponding to cell 810. One such fieldlabel-value pair that may be detected in cell 810 has a field label of“productid” and a value of “SF-BVF-G01.”

If a user selects a column in the table interface, which couldcorrespond to the selection of column 804 a shown in FIG. 8C, the systemcould attempt to detect at least one field label-value pair that is atleast partially within the data items corresponding to column 804 a. Asa further example, if a user selects multiple columns in the tableinterface, which could correspond to the selection of columns 804 c and804 e shown in FIG. 8D, the system could attempt to detect at least onefield label-value pair that is at least partially within the data itemscorresponding to columns 804 c and 804 e.

Additionally, if a user selects a portion of a textual representation ofa values of one or more data items in the table interface, which couldcorrespond to the selection of portion 814 of textual representation 812shown in FIG. 8F, the system could attempt to detect at least one fieldlabel-value pair that is at least partially within portion 814. One suchfield label-value pair that may be detected partially within portion 814has a field label of “productid” and a value of “BS-AG-G09.” The usermay select at least a portion of text corresponding to the field label,the value, or both the field label and value. In some cases, the usermay optionally be required to select all of the associated text and notjust the portion.

In further respects, each field label-value pair may be detected basedon determining a corresponding extraction rule. For example, asindicated above, a field label-value pair may be detected based on beingextractable using the corresponding extraction rule. However, in orderto detect the field label-value pair, the field label-value pair doesnot necessarily have to be extracted using the corresponding extractionrule (although it may be). Rather, the system need only detect that afield label-value pair is extractable at least partially within theselection portion using the corresponding extraction rule. A fieldlabel-value pair may be considered at least partially within a selectedportion of a data item for purposed of detection where at least some ofthe textual representation of the data item could be utilized togenerate the value and/or the field label of the field label-value pairusing the corresponding extraction rule.

The corresponding extraction rule could be a predefined extraction rule,a user generated extraction rule, a user specified extraction rule, oran automatically generated extraction rule, as some examples. In somecases the extraction rule may be generated at least partially based onthe selection of the portion of the data items. In these cases, theextraction rule may be automatically generated by the system, or may beautomatically generated and subsequently revised based on user input. Asan example, an extraction rule can comprise one or more regularexpressions that provide instructions for identifying field labelsand/or values from data. As another example, an extraction rule caninclude detecting a first text portion as being separated by one or moredesignated demarcating characters from a second text portion in data.

Further based on the selection of the portion of the data items, thesystem can cause display of one or more options corresponding to one ormore of the detected field label-data item pairs. For example, theoptions could be included in an option menu, which could be one ofoptions 930 a, 930 b, 930 c, or 930 d or option menu 926 of FIG. 9 basedon detection of a field label-value pair at least partially in aselected portion of a table format (e.g., cell(s), column(s), and/orportion(s) of a textual representation(s)). With respect to FIG. 8F, theoption could be option 832 and could read “Extract ‘productid’,” with‘productid’ being contextually generated based on the user selection. Inaddition, or instead, one of the options could read “Autoextract,” andcorrespond to the autoextract command described above.

It is noted that not all detected field-label value pairs may besuggested with the options. Instead, a subset of detected field-labelpairs may be suggested for extraction. Various selection criteria couldbe employed to choose which detected field-label pairs to suggest forextraction. In some cases, the selection criteria may be based on anumber of field label-value pairs that are extractable using thecorresponding extraction rule, with one or more detected fieldlabel-value pairs corresponding to the top numbers being used forsuggestion.

Based on an option of the displayed one or more options being selected,the system can carry out one or more operations associated with theoption. In some cases, the operations comprise one or more data itemsbeing assigned to one or more fields of the set of events (e.g., a newfields and/or existing fields). The one or more data items can beextracted using the corresponding extraction rule(s) for the one or moreof the detected field label-data item pairs that correspond to theoption.

At least one of the one or more data items may optionally comprise avalue extracted and/or displayed before the option was selected, forexample, as part of detecting the at least one field label-value pairs.However, at least one of the one or more data items may optionallycomprise a value extracted based on, or responsive to, the selection ofthe option. For example, as described above, detection of a fieldlabel-value pair does not require extraction by the correspondingextraction rule, but could use some heuristic. Another scenario could bethat the corresponding extraction rule was only partially executed fordetection, for example, until at least one field label-value pair hadbeen extracted. Yet another example is where the option corresponds to acommand that extracts field label-value pairs that are identified withinmore than just the selected portion(s) of the data item(s), such as anentire column, where the user selected a cell or a textualrepresentation of a data item value.

Thus, a user could select the “Extract ‘productid’” option describedabove, and the system could assign only field label-value pairs thatinclude the field label “productid” and any associated extracted valuesto data items of events as a new field that the system assigns the fieldlabel of “productid” (e.g., using a command associated with the selectedoption). The system may further automatically add a column correspondingto the new data items to the table format, or other format utilized todisplay events. As another example, a user could select the“Autoextract” option described above, and similar operations may beperformed as the Extract ‘productid’” option, with additional operationsbeing performed to extract any other field label-value pairs that anautoextract command discovers at least partially in the selectedportion(s) of the data item(s).

It is noted that options that are displayed and correspond to one ormore of the detected field label-data item pairs need not be displayedin an option menu, or in the same option menu as other options whenimplemented with search screen 800. In some cases, one or more optionsmay be displayed in a sidebar, such as sidebar 830, as one example. Asanother example, one or more of options may not be selectable. Forexample, an option could be displayed as non-selectable dialog, and maybe included in a dialog box. The user may select the one or moreportions of the events by hovering over a selectable region thatcorresponds to the one or more portions of the events. If a fieldlabel-value pair is detected, the dialog may be presented to the user.

It is further noted that although suggesting field label-value pairextractions could be integrated into a search interface that correspondsto search screen 800, with the set of events being search resultsdescribed with respect to search screen 800, field label-value pairextractions could be suggested in other contexts, and need to be basedon search results. As an example, the suggestions could be provided aspart of a configuration interface (or search interface), where an optionmay be operable to save the corresponding extraction rule(s) (e.g.,generated based on the user's selection of events) to a configurationfile. In addition, or instead, an option may save the data itemsextracted with the corresponding extraction rule(s) (e.g., to theconfiguration file). The configuration file could be used to apply thesuggested field label-value pair extractions (e.g., the correspondingextraction rule(s)) to other events than were utilized to generate thesuggestions. For example, the configuration file may be loaded for thispurpose in a search interface (e.g., in search screen 800). These andother possibilities are contemplated with respect to suggesting fieldlabel-value pair extractions.

3.1 Additional Features

FIGS. 12A, 12B, 12C, 12D, 12E, 12F, and 12G (also referred to hereincollectively as “FIGS. 12”), show a progression of a search screen in anexemplary search interface through user interaction with a table formatin accordance with some implementations of the present disclosure. Theprogression is in the depicted sequence shown, but at least some of theprogression is not shown.

The search interface utilizes the search screen to display one or moreevents returned as part of a search result set based on a search query.The search interface can correspond to the search interface describedwith respect to search screen 800 of FIG. 8A. Unless specified, searchscreen 1200 and the search interface offer similar functionality assearch screen 500 and the search interface described above. Inparticular, the following description if presented to provide additionalpotential features that can optionally be incorporated into searchscreen 800 and its corresponding search interface, and not to limitfeatures to certain implementations. As with the description abovesections, it should be appreciated that various concepts described belowhave more general utility than for search interfaces, or for aparticular type of search interface. In these respects, various conceptsare severable from the particular implementations described herein. Asone specific example, although various concepts are described hereinwith respect to search, many of these concepts are more broadlyapplicable to queries in general. Thus, while the terms searchinterface, search screen, search results, search query, and othersimilar terms are utilized herein, these concepts are more broadlyapplicable to query interfaces, query screens, query results, andqueries. Types of queries include search queries, script queries, anddata processing queries.

3.2 Interface Panel

As shown in FIG. 12, the search screen includes interface panel 1205. Invarious implementations, interface panel 1205 is purpose-built forassisting users in formulating commands for queries, such as the querycorresponding to the search results displayed in table format 1202. Inthe implementation shown, the query is represented in sidebar 1230,corresponding to command entry list 1208.

Interface panel 1205 is adjacent sidebar 1230 and extends lengthwisealong sidebar 1230. However the particular location, size, and relativepositioning of interface panel 1205 can vary. For example, interfacepanel 1205 could extend crosswise in a search screen (e.g., adjacent thetop or bottom of the search screen). Furthermore, in the searchinterface, interface panel 1205 is anchored in its position, but inother implementations could be floating and movable by the user.Additionally, although a single panel is shown, in some cases,functionality of the interface panel could be incorporated into multiplepanels, which could be shown concurrently and/or could be independentlyhidden and unhidden. It should be appreciated that any of the variousaspects of interface panel 1205 could be independently configurable by auser.

In the search interface of FIG. 12, interface panel 1205 is implementedas a panel, by way of example only. In other cases, a window, tab, orother interface mechanism(s) can be employed. A user can selectivelyhide and unhide interface panel 1205 by selecting form element 1207,which is implemented as a clickable button. For example, FIG. 12D showsinterface panel 1205 in a hidden state (e.g., partially or fully hidden)and the remaining ones of FIG. 12 show interface panel 1205 in anunhidden state. Interface panel 1205 could be hidden or unhidden in anyof the various FIG. 12 shown. In implementations where the user can hideand unhide interface panel 1205, the user is able to selectively accessthe various form elements included therein as needed for adding toand/or modifying the search query, as is further described below.

As shown, interface panel 1205 comprises one or more form elements toassist the user in constructing the query. For example, in FIG. 12A,interface panel 1205 includes form elements 1209, in FIG. 12B interfacepanel 1205 includes form elements 1211, in FIG. 12C interface panel 1205includes form elements 1213, in FIG. 12E interface panel 1205 includesform elements 1215, in FIG. 12F interface panel 1205 includes formelements 1217, 1219, and 1221, and in FIG. 12G interface panel 1205includes form elements 1223.

3.3 Interface Templates

From FIG. 12 it should be apparent that the form elements and contentsthereof included in interface panel 1205 can change over time based oncontext to assist the user in interacting with table format 1202. Invarious implementations, the present disclosure provides for interfacetemplates for populating interface panel 1205. As indicated aboveinterface panel 1205 could be implemented utilizing multiple panels,windows, and/or tabs, and need not be a single panel as shown.Furthermore, at least some interface templates may correspond to one ormore dedicated interface panels, windows, and/or tabs.

Each interface template generally comprises instructions for one or moreforms. In some implementations, each interface template corresponds to arespective table manipulation action a user may perform on table format1202. As used herein, a table manipulation action may describe apredefined alteration to the number of rows, columns, and/or cells in atable format and/or the contents thereof that can be achieved by addingone or more commands to a query, such as the search query represented bycommand entry list 1208.

In some implementations, each option presented to the user throughselection of one or more interactive regions (e.g., selectable cells,columns, rows) of the table format can correspond to a tablemanipulation action and interface template. For example, the options canbe the options in option menu 926 or any of the various optionsdescribed herein. Where an option is instantiated in the searchinterface, it is done so as instructed by its associated interfacetemplate. For example, an interface template can provide instructionsthat define any of the various features of an option described withrespect to FIG. 9, including context for displaying the option based onthe selection that prompted option menu 926 (or more generally the listof options).

Further, the interface template for an option defines the overall flowof user interaction with the option. In various implementations, theoverall flow of user interaction defined by an interface templateextends from the presentation of the option through modification of thequery (e.g., through adding one or more commands to the search querybased on user selection of the option). In this way, an interfacetemplate can define one or more forms and the sequence and/or conditionsfor presentation of the one or more forms (e.g., whether a form is aform-based option, nested form-based option or any of features of anoption). Further, an interface template can define one or more formelements for each of the forms, the number thereof to include in a form,as well as the contents thereof, including potential default contents.Various examples of such contents have been described herein.

Additionally, an interface template can define one or more mappingsbetween form elements and modifications to the query. For example, aninterface template may map form elements to one or more commandidentifiers and/or command elements for one or more commands to add tothe query. In doing so, the interface template defines the syntax forthe one or more commands. Thus, as has been previous described herein,the one or more commands that are added to the search query may be inproper syntax for the search query, complete with command identifiersand any command elements that are needed or desired for execution of thecommands. Thus, the user can perform the table manipulation actionassociated with the interface template with little to no knowledge ofthe underlying query language.

In various implementations, in response to a user selecting an option,the one or more forms defined by the interface template are displayed tothe user. For example, in FIG. 12, the one or more forms are displayedin interface panel 1205. Where a different form is displayed ininterface panel 1205, that form may be replaced with the one or moreforms. As an example, after selecting option 1220 a in FIG. 12A, theform comprising form elements 1209 is automatically replaced with a formcomprising form elements 1211, as shown in FIG. 12B. As shown, the formelements include a text box comprising an evaluation function labeled“strftime,” which is the evaluation function applied by the “eval”command Below the text box are other form elements that each correspondto a respective evaluation function that is compatible with the “eval”command. The form elements are individually selectable to cause therespective evaluation function to be entered into the text box. In someimplementations, the respective evaluation function is provided enteredas a syntax template with the function identifier and one or morevariables that the user can replace with attribute labels and/or valuesusing the text box. In addition, or instead, the evaluation functionentered could be generated based on the data items and/or eventattributes corresponding to the interactive region (and optionally thedata type) selected by the user to display an option selected to displaythe form. For example, for a cell selection, the “X” in form elements1211 could automatically be a value of the data items of the cell whenentered into the text box. Generally, one or more values of selecteddata items or values generated therefrom and/or event attributeidentifiers corresponding to the selected data items could beautomatically mapped to the evaluation function templates. All of thisbehavior could be defined by the instructions of an interface template.

Optionally, an interface template can include instructions for causingdisplay or causing removal of display of the one or more forms definedby the interface template. For example, an interface template mayinclude instructions for unhiding interface panel 1205 based onselection of the option. Other interface templates may not include suchinstructions. Thus, for example, if interface panel 1205 had beeninitially hidden in FIG. 12A, the one or more forms defined by theinterface template may not be displayed to the user. In contrast, wherea user selects option 1220 b in FIG. 12B, interface panel 1205 maydisplay a form comprising form elements 1213, as shown in FIG. 12C,regardless of whether interface panel 1205 was initially hidden in FIG.12B by automatically unhiding interface panel 1205 based on selection ofoption 1220 b when interface panel 1205 is hidden.

Automatically unhiding interface panel 1205 and/or displaying the formcan be beneficial, for example, where user input is desired forconstructing the one or more commands to add to the query. One suchexample could be, for option 1220 b, corresponding to a “rename valuesoption” that enables a user to specify new values for particular valuesof event attributes in table format 1202. Each form element comprises atext box having a default value generated from a respective value of anevent attribute selected based on an analysis of the selected column (orin other cases cell or combinations thereof). In some cases, each valuecan be included in the form with a respective form element. Otherapproaches include selecting a subset of the values, such as the topfive values and/or presenting a dropdown list of any of the variousvalues in the selection portion of the table format for a particulartext box. In order to change the values, the initial values aredisplayed to the user and the user is able to define replacement valuesfor the displayed default values via input into the form elements. Byclicking the apply button, the query is updated to carry out the tablemanipulation action. For example, the one or more commands areautomatically constructed and/or updated to carry to rename the valuesin in the selected event attribute. As shown, the event attribute isautomatically detected based on the user selection of the column thatcorresponds to the event attribute. The event attribute canautomatically be included as a command element in a command to specifywhich event attribute should have its values renamed. An example of thecommand that may be generated by the system is: “activity”=case(activity==“Download”, “Enterprise”, activity==“Universal ForwarderDownload”, “Universal Forwarder”, activity==“Splunk Light Download”,“Splunk Light”, activity==“Mobile Access Server”, “Mobile AccessServer”, activity==“Hunk Download”, “Hunk”). It will be appreciated thatall of the foregoing may be specified by the interface template.

In further respects, in addition to or instead of each optioncorresponding to a table manipulation action and interface template,each command entry may correspond to a table manipulation action andinterface template. For example, FIG. 12 shows command entry list 1208,which can correspond to command entry lists 808, 1008 a, and 1008 b,described above. In various implementations, each command entry list canbe selected to cause one or more forms to be displayed, as instructed byan interface template. In some cases, a user selects a command entry soas to access one or more forms for modifying the one or more commandsrepresented by the command entry. As an example, a user may select acommand entry by clicking on the command entry or a portion thereof.Clicking on a command entry may be similar to clicking on an option inthe option menu, as described above. In particular, one or more formsmay be displayed in a similar manner. The form elements of the one ormore forms can be populated with at least a portion of the one or morecommands represented by the command entry, such as one or more commandelements. Further the one or more commands can be modify by the user viainput to the one or more form elements. Also similar to an option, theone or more forms may be displayed in interface panel 215. This behaviorcan be defined by an interface template.

Furthermore, for the purpose of displaying the one or more forms of theinterface template, in some cases, only one command entry may beselected at a single time. Selecting one command entry may automaticallydeselect the currently selected command entry and update the displayedform. When interface panel 215 is hidden upon selection, the form mayremain hidden until being unhidden by the user.

In addition to or instead of selecting a command entry directly, acommand entry may be automatically and indirectly selected based on auser selecting a corresponding option, such as one of the options in anoption menu. More particularly, selection of an option may cause acorresponding command entry to be added to command entry list 1208(e.g., to the bottom of the list) representative of the one or morecommands added to the query. Additionally, adding the command entry maycoincide with the command being selected in command entry list 1208.Thus, a single interface template may correspond to both an option and acommand entry that may be created as part of the interface template. Insome cases, there is a one-to-one correspondence between an option and acommand entry. Furthermore, there may be a one-to-one correspondencebetween a command entry and a command represented by the command entry(i.e., one command per command entry).

In various implementations, selection of a command entry in the commandentry list causes the one or more forms of the interface template to bedisplayed in the same state as when the user completed previousinteraction with the forms of the associated interface template. Thus,the previous form may be displayed to reflect the various user inputinto the form elements, such as user selections and user entered text.In this way, the user can go back to the forms via the commend entry asneeded to modify the one or more commands using the same forms and/orinformation entered by the user. It should be appreciated that the oneor more forms may or may not be the forms used to initially generate theone or more commands after selection of a corresponding option.

In various implementations, the current state of the various formsaccessed in FIG. 12 can be stored as metadata. For example, the metadatamay be saved in association with a saved pipeline (which may also bereferred to as a data object, a search object, or a query object). Inthe example of FIG. 12, this saved pipeline may correspond to savedpipeline label 1104 c in FIG. 11 that reads “MyReport.” Where a userloads the saved pipeline into a search interface, such as by selectingedit pipeline link 1108 c, the associated metadata may be loaded as welland the search interface can be configured as instructed by themetadata. Thus, the user can continue interacting with the query wherethe user left off at save time. Saving and loading the metadata can beuseful for various reasons. One benefit is that the system can save andreuse relevant information about a query or query session that is notobtainable from the query itself. Another benefit is even where theinformation is obtainable from the query, the system does not need toprocess the query to extract the information or be coded to perform suchcomplex extraction.

Other metadata that can optionally be saved in association with apipeline include modifications made by the user to table formatting. Asan example, the user may rearrange the ordering of the displayed columnsin table format 1202. Column format metadata can describe the orderingof columns such that it is preserved when the saved pipeline is lateraccessed. In FIG. 12A, option 1220 c may be used to reorder a column.Option 1220 c does not cause one or more commands to be added to thequery. Instead, option 1220 c causes the metadata to reflect theordering of the columns as modified by the user based on selecting theoption. Using option 1220 c, the user may place column 1204 a wheredesired, such as by swapping places with column 1204 b. As a furtherexample, column format metadata can capture column widths that the usermay modify for any of the various columns, for example, by draggingcolumn dividers. Other optional column formatting metadata includecolumn data types and/or column names. This and other table formattingmay be stored in the metadata.

Also, as described above in section 2.6 a query of a saved pipeline maybe dependent on one or more other queries that each may be saved as arespective saved pipeline. The metadata of each saved pipeline maycorrespond to the portion of the query that was composed for and savedto the saved pipeline (e.g., the extended portion or an input portion)and loading of the query using the saved pipeline could optionally loadthe metadata (or portions thereof) of each saved pipeline from which isdepends. In some cases, a saved pipeline includes at least one link to asaved pipeline from which it depends. As an example, a saved pipelinemay include a link to the saved pipeline it extended, such that a chainof links may be formed amongst saved pipelines back to an initial inputpipeline. The chain of links can be used to sequentially load the chainof saved pipelines starting from the initial input pipeline andoptionally the metadata associated with the respective saved pipeline.However, it is noted that this is one example and the metadata need notbe saved with respect to a particular portion of the query or otherwisebe specific to a saved pipeline.

3.4 Supplemental Event Attributes in a Table Format

In various implementations, the present disclosure relates to addingsupplemental event attributes to a table format that includes eventattributes from a query. In particular, one or more supplemental eventattributes, and data items corresponding to the one or more supplementalevent attributes can be added to the table format in one or morecorresponding supplemental columns. The data items of supplemental eventattributes (i.e., supplemental data items) are incorporated into thetable format and are from at least one source external to the initialquery. In particular, the supplemental data items are included in eventsthat correspond to results of the initial query. In doing so, the useris able to utilize the table format to interact with data itemscorresponding to the initial query along with the supplemental dataitems. For example, table manipulation actions available to the userthrough interactions with supplemental data items and supplemental eventattributes may be indistinguishable from their counterparts thatcorrespond to the initial query. Thus, the user can continue interactingwith the table format without concern for the source of the data items.

The supplemental data items for a supplemental event attribute can bedefined by a data object. In particular, a data object may definepotential data items for one or more event attributes for the events. Insome cases, the potential data items are statically defined, such as inone or more a table files. For example, the potential data items andoptionally event attributes may be statically defined in a tabledefinition file, such as a comma-separated values (CSV) file, a keyholemarkup language (KML) file, an extensible markup language (XML) file, orgenerally any file or format that represents values for the potentialdata items.

In other cases, the potential data items are dynamically defined by oneor more instructions that are executable to derive potential data itemsand/or event attributes for the table format. One example of a dynamicdefinition includes a query, such as a search query, that can beexecuted (by the system or externally) to produce the potential dataitems and/or event attributes. As an example, a data object comprising adynamic definition could correspond to one of the saved pipelinesrepresented in FIG. 11. The system could retrieve potential data itemsand/or event attributes as query results. However, the executableinstructions need not include a query. Examples of instructions includea script, query, code, and/or binary executable that can provide thepotential data items and/or event attributes.

The data utilized to produce supplemental data items based on staticdefinitions and/or dynamic definitions can be stored within the systemand/or external to the system. In various implementations, thedefinitions can specify a location for the system to access the data. Alocation can comprise a uniform resource identifier (URI), an internetprotocol (IP) address, a domain, a file system path, or otherinformation defining a location to access the data.

In some respects, the system can be used to create and/or preconfiguredata objects for providing supplemental event attributes to a tableformat by a user. For example, the user can configure a data object andthe potential data items and/or event attributes defined by the dataobject that may be added to the table format in a user interface. Insome cases, the user could define a subset of available potential dataitems and/or event attributes that can be added to a table format usingthe data object. Data objects may further be assigned permissions,owners, and/or associated applications to determine which users and/orapplications (e.g., a search or query application) are able to accessand modify data objects.

3.5 Accessing Data Objects to Add Columns to a Table Format

Other users can access the preconfigured data objects in the searchinterface subject to their configurations so as to add supplemental dataitems therefrom to a table format. Referring to FIG. 12F, an exemplaryinterface for adding supplemental event attributes to table format 1202is shown. By way of example, a user may initiate adding supplementalevent attributes to table format 1202 by selecting option 1226, whichcan be a button in sidebar 1230, as shown. However, there are many waysthe option could be presented, such as in an option menu (e.g., optionmenu 926) and/or contextually.

Option 1231 corresponds to an interface template, which in response toselection by the user, causes display of one or more forms in the searchinterface (e.g., in interface panel 1205). As shown in FIG. 12F, one ofthose forms includes a list of data objects (e.g., preconfigured dataobjects) available as sources of supplemental event attributes to tableformat 1202. The system may include additional data objects, butoptionally, the data objects shown are listed based on the present userhaving permission to access the data objects.

For the present example, assume the user selects link 1228 (or option)corresponding to the data item labeled “SupplementalData” from formelements 1217. In response, another form of the interface template isoptionally presented to the user (e.g., in interface panel 1205) alongwith a list of reference event attributes that can be used by the systemto add supplemental event attributes from the selected data object. Thelist of reference event attributes correspond to event attributes ofevents in the table format. A reference event attribute represents anevent attribute of the events to use as a reference for selectingsupplemental data items to add to the table format.

How a reference event attribute is used as a reference depends on thecombination type used for adding the supplemental data items to thetable format. Various combination types may be employed, such as alookup, a join, an inner join, an outer join, a left outer join, a rightouter join, a full outer join, a natural join, or a merge. A lookup canlookup values of the data items of the events in the reference eventattribute and assign the results of the lookup to the events as one ormore supplemental event attributes. As indicated above, a lookup can bedefined by the data object selected by the user. In cases wherepotential data items are statically defined, a static lookup table maybe employed. In cases, potential data items are dynamically defined, thedata object can provide transforms for producing supplemental data itemsfor the table format from the values of the reference data items.

As another example, a join may utilize the reference event attribute asa foreign key for the join. However, as the join is not performed onrelational data, but instead query results, there is no true foreign keyrelationship established between the data sets that are joined. Forexample, a join may combine the query results of the query with queryresults of a subquery (e.g., a saved pipeline) (e.g., if specified eventattributes are common to each). As another example, a join may combinethe query results with relational data, such as a relational databasetable. In these cases, the system may match event attributes to columnsof relational database tables.

Thus, supplemental event attributes may be added to query results of thetable format from relational database tables and/or query results.However, depending on the combination type being employed, certain eventattributes may not be suitable for serving as a reference eventattribute. As an example, for a join, the data object may not define anysuitable event attribute for combination with one or more of the eventattributes of query results. In some respects, the system analyzes theevent attributes of the query results and/or the potential eventattributes defined by the data object to identify those that aresuitable for the particular combination type.

For a join, the system could analyze attribute labels of attributes(e.g., relational or late-binding) and/or values thereof (e.g., ofrelational values or of data items) to determine whether a particularattribute matches one or more event attributes in the query resultsdisplayed in the table format (e.g., could serve as a foreign key for ajoin). Various factors may be employed, alone or in combination, toidentify one or more reference event attributes based on comparisonsbetween event attributes of the query results and attributes defined bya data object. In some cases, a score may be generated (optionally foreach factor to create an aggregate score) that quantifies a likelihoodthat an event attribute of the query results should be used as areference event attribute. However, certain factors alone may besufficient for qualifying an event attribute as a reference eventattribute. One example is where an attribute label of the eventattribute is identical to an attribute label defined by a data object.However, in other cases, a similarity score could be generated.

Another potential factor relates to a similarity between values of thedata items of the event attribute of the query results and values of theattribute defined by the data item. For example, where the systemdetermines the values of the same data type or substantially the samedata type (e.g., categorical or numerical), the system increases theconfidence that the event attribute could serve as a reference eventattribute. In addition, or instead, the system may analyze similaritiesin the information type being represented by the data items. Forexample, the system may determine that both attributes represent e-mailaddress, phone numbers, IP addresses, and/or URLs. The more similar theinformation types, the higher confidence the system can have that theevent attribute could serve as a reference event attribute. The systemcould further identify matches between particular values from each theattributes. The more matches (e.g., exact or substantial matches), themore likely the event attribute should be used as a reference eventattribute.

The list of reference event attributes shown with respect to formelements 1219 in FIG. 12F include one or more reference eventattributes, potentially identified and/or selected by the system asdescribed above. In the example, shown, only the event attribute havingthe attribute label salesforce_id displayed in the table format wasselected as an event attribute. For example, the event attribute mayhave been selected by the system based on the data objectSupplementalData comprising salesforce_id in an attribute label.Different or additional event attributes could have been presented inthe list had the system identified those as reference event attributes.For example, the system may have selected from event attributescorresponding to _time, date, productDownloaded, Platform, andsalesforce_id, shown in FIG. 12E. The event attributes may have beenselected from based on being displayed in the table format. Optionally,the system may also select from event attributes that are not displayedin the table format, but are still associated with the events in thetable format.

The user can select one or more of the displayed reference eventattributes for the combination with the data object. In response to theuser selecting, for example, salesforce_id in FIG. 12, the systemanother form of the interface template is optionally presented to theuser (e.g., in interface panel 1205) along with a list of supplementalevent attributes that can be used by the system to add supplemental dataitems from the selected data object using the selected reference eventattribute(s). The system can identify one or more supplemental eventattributes from the data object based on the reference eventattribute(s) selected by the user. In particular, the supplemental eventattributes can correspond to event attributes that can be incorporatedinto query results corresponding to the query based on a selectedreference event attribute (e.g., for a given combination type).Referring to FIG. 13 with FIG. 12F, FIG. 13 shows an exemplary searchscreen 1300 in accordance with disclosed embodiments. The searchinterface comprises the saved pipeline corresponding to theSupplementalData data object selected by the user in FIG. 12F. Based onthe user selecting a reference event attribute correspondingsalesforce_id, the system can identify supplemental event attributeshaving attribute labels raw_eloquaGUID, raw_version, architecture, andclient_ip from the data object as candidates for the combination. Asupplemental event attribute corresponding to salesforce_id is excludedbased on the user selecting a corresponding reference event attribute.

As indicated with respect to form elements 1221, the user has selectedsupplemental event attributes corresponding to raw_eloquaGUID andarchitecture for adding to the table format. Based on the user selectingthe supplemental event attributes, the supplemental event attributes areadded to the table format. For example, the adding may be in response tothe user selecting apply button 1232.

FIG. 12G shows supplemental columns 1233 a and 1233 b added to tableformat 1202 in response to the user selecting the supplemental eventattributes using form elements 1221. The table format is caused to beupdated to include supplemental columns 1233 a and 1233 b correspondingto the supplemental event attributes selected by the user using formelements 1221 and populated with the potential data items defined by theselected data object as supplemental data items. Had the user onlyselected raw_eloquaGUID, only supplemental column 1233 a would have beendisplayed. Further, additional supplemental columns would be displayedhad the user selected additional supplemental event attributes.

In further respects, a supplemental column may be added to the tableformat based on metadata corresponding to the attribute of thesupplemental column associated with the data object. In particular, eachdata object can be preconfigured with metadata, such as has beendescribed above with respect to search screen 1200. For attributescorresponding to supplemental columns added to the table format,corresponding column format metadata can be incorporated into the tableformat. Thus, the supplemental columns can be added based on the columnwidth and column ordering corresponding to the attributes. The samewidths and column ordering may be used, with each column being added toone side of the existing columns in the table format, as shown. Theextracted metadata may optionally be stored in association with one ormore commands (e.g., a command entry) used to cause the column to beadded and may supplement metadata describing the previous state of thetable.

Also shown, FIG. 12G, similar to other options described above,selecting the option caused a command entry a corresponding commandentry (command entry 1234) to be added to command entry list 1208. Thus,it should be appreciated that the interface template could have any ofthe various features previous described for interface templates. Forexample, selecting the option can cause one or more commands to be addedto the search query to execute the selected combination. Further, theuser can select command entry 1234 to cause the form corresponding toform elements 1221 to reappear where the user may reselect supplementalevent attributes. Further the user can optionally navigate back throughthe forms of the interface template and change any of the variousselections made. The updated selections can automatically be reflectedin the displayed table format (e.g., based on the user selecting applybutton 1232 to apply the updated selections). Similar to other commandentries, the command entry and one or more commands represented by thecommand entry can be automatically updated to correspond to updatedselections made by the user.

The presence of at least one supplemental event attribute indicates tothe system that potential data items exist for the table format in adata object. However, in other cases, no supplemental event attributesmay be identified for a data object. Thus, the system may refrain frompresenting the particular data object to the user using the interfacetemplate. For example, the system could identity all potential referenceevent attributes for the data object (e.g., for a given combinationtype) and determine whether at least one supplemental event attributeexists for the table format based on the potential reference eventattributes. Data objects may be excluded from form elements 1217 wherethe system is unable to identify at least one supplemental data item forthat data object. Thus, in some cases, whether a data object ispresented to the user in form elements 1217 is based on theidentification of one or more supplemental event attributes for thetable format from the data object.

Having added the supplemental columns to the table format, the user cancontinue to perform table manipulation actions of the table format. Tothis effect, the supplemental event attributes and data items thereofmay be treated identically from the perspective of the user'sinteractions with the table format. Thus, the user need not be concernedwith the source of the data items displayed in the table format. Forexample, as shown in FIG. 12G, the user has selected interactive regionscorresponding to columns 1233 a, 1233 b, and 1233 c. Despite columns1233 a and 1233 b being supplemental columns, the same option menu andoptions are displayed based on the selection as would have beendisplayed had each column been based on the source of the initial query.

As indicated above, various combination types may be available foradding supplemental event attributes to the table format. However, someusers may not be familiar with the differences between the combinationtypes. Furthermore, the some users may be unfamiliar with the particularsyntax for the combination types, which could be significantly differentper combination type. In various implementations, using the interfacetemplate corresponding to option 1226, the user is able to addsupplemental event attributes to the table format without extensiveknowledge of the combination types and/or query language. From theuser's perspective, the user may be selecting available columns of datato add to the columns in the table format, with the system managing theunderlying complexities of identifying and combing data sets, which canbe from vastly different sources, such as query results, late-bindingschema data, relational data, lookup tables, and the like.

In some implementations, different data objects correspond to differentcombination types. For example, one data object may be for a join andanother may be for a lookup. This information may optionally bepreconfigured in metadata of the data objects by the user and/or thesystem. For example, saved pipelines may be suitable for joins whereaslookup tables may be suitable for lookups. The suitable reference eventattributes and/or supplemental event attributes can optionally beidentified and/or selected based on these combination typesautomatically, such that the user need not be concerned with differencesbetween the combination types. Thus, with respect to format elements1217, SupplementData could correspond to a join whereas MyLookup couldcorrespond to a lookup. The user may be presented with a similarinterface for selection of either data object and a suitable command(s)will be generated by the system for the query.

It will be appreciated that in some cases, the system could allow theuser to select the desired combination type. For example, prior to theform comprising form elements 1217, the user could be offered to selectthe combination type and the data objects in form elements 1217 may bebased on the selected combination type.

3.6 Selecting Command Entries to View Corresponding Query Results

As indicated above, in various implementations, a user may select acommand entry in a command entry list to view query resultscorresponding to the selected command entry. In particular, the user mayselect a command entry to specify an endpoint in the query representedby the command entry list. By selecting an endpoint, the query resultsdisplayed in the table format are caused to correspond to the commandsof the query up to the endpoint. In some implementations, by selecting acommand entry, the endpoint is specified as being directly after the oneor more commands represented by the selected command entry. Thus, forexample, a user may select command entry 1233 in FIG. 12G to specify the“join” command as the endpoint.

It should be appreciated that each command entry in the command entrylist may be similarly selectable to achieve a similar result for the oneor more commands it represents. Further, when command entries are addedto the command entry list, those command entries are also selectable. Acommand entry may be selected, for example, by clicking on the commandentry in the command entry list. By selecting a command entry the queryresults displayed in the table format may optionally be automaticallyupdated to correspond to the query up to the specified endpoint. Thus,the user can view the progression of the query results and by selectingthe various command entries in the command entry list without losingsubsequent commands of the query. It is noted that selecting a commandentry typically automatically deselects the previously selected query.Furthermore, when a command entry is added to the command entry list, insome cases, the command entry becomes automatically selected.

In some cases, when the query results are updated to correspond to thequery up to the endpoint, the corresponding commands of the query areexecuted to update the query results. By way of example, by selectingcommand entry 1233 in FIG. 12G, the table format may look similar towhat is shown in FIG. 12E (although if commands of the query arecompletely re-executed, the events in the query results could differ).It is noted that the command entry list may still look as it does inFIG. 12G except that command entry 1233 could be grayed out instead ofcommand entry 1231 to visually indicate the currently selected commandentry to the user. By subsequently reselecting command entry 1231,search screen 1200 would look similar to how it does in FIG. 12G.

In further respects, the user may optionally be permitted to interactwith the displayed query results corresponding to the query up to theendpoint to insert one or more commands into the query directly afterthe endpoint, and to insert one or more command entries that representthe one or more commands directly after the selected command entry inthe command entry list. For example, the user may select interactiveregions (e.g., cells, columns, text) of the table format to add one ormore commands to the query as has been described throughout theapplication.

In some cases, the user may optionally be precluded to interact with thedisplayed query results where the query results do not correspond to theentire query. Also, as one example, the selection of prior commandentries in the command entry list may be utilized to preview priorstates of the query results (e.g., without completely re-executing thecommands). In some cases, the prior states of the query results could besaved with respect to a corresponding command entry so that the commandsneed not be completely re-executed on the data sources. However, atleast some of the commands may be executed as needed to accuratelyportray the query results (e.g., after inserting a command entry and/ormodifying one or more commands thereof).

It is also noted that in the present implementation, selecting a commandentry displays a corresponding form allowing the user to modify the oneor more commands represented by the selected command entry using one ormore form elements of the form. As has been described previously, thisform could be the same form displayed to the user when the one or morecommands where adding to the query or modified in the query.

It is further noted that in some implementations, metadata can be storedwith respect to a particular command of the query and/or command entry.For example, a state of the search interface can be saved to eachcommand entry that represents the state when the command entry waspreviously selected (e.g., first added and/or last modified). The statein the metadata can include the column formatting (e.g., column widths,data types, orderings, and/or names) for each column in the table andoptionally other information such as user input into one or more formscorresponding to the command entry. Thus, for example, when a newcommand entry is selected, the metadata of the command entry can beloaded into the table. Furthermore, any changes made while the commandentry is selected can be reflected in the metadata of the command entry.In some cases, one or more changes are propagated to the metadata of oneor more other command entries (e.g., each subsequent command entry inthe list). As an example, column width and/or data types of columns maybe propagated to the metadata. In some implementations, a change tocolumn names may be propagated through the subsequent commands of thequery. For example, a column name may comprise an attribute label andcommands may reference event attributes by their attribute label. Thus,changing an attribute label may results in the references beingautomatically updated with the changed attribute label.

While the present example is given with respect to the table format ofFIG. 12G, the query results need not be displayed in a table format, orcould be displayed in a different table format than described withrespect to search screen 1200. In particular, it is emphasized that thisconcept may be implemented in any of the query interfaces describedherein, or in other interfaces.

3.7 Integrating Multiple Query Interfaces

The present application has described multiple search screens, which cancorrespond to different query interfaces. For example, the presentapplication has described search screens 600, 800, 1200, and 1300. Thevarious query interfaces offer different feature sets and may be moresuitable for different types of interactions based on those differences.For example, the search interface corresponding to search screen 600 maybe more suitable for filtering query results based on specified eventattributes and/or values for the query results. The search interfacecorresponding to search screen 1200 may be more suitable for identifyingspecific information in filtered query results. For these and otherreasons, including personal preferences, users may desire to utilize thedifferent query interfaces for different purposes. In some respects, thepresent disclosure relates to integrating query interfaces, such thatusers can maintain progress made on constructing a query in one queryinterface in another query interface. Thus, users can construct variousportions of the query in different query interfaces depending on theirparticular needs.

In some implementations, a query previously saved and/or constructedusing the search interface corresponding to search screen 1200 (orsearch screens 800 or 1300) may be used as an input to the searchinterface corresponding to search screen 600. For example, as describedabove, in response to a user selecting a use as input link in FIG. 11,the system may cause the corresponding pipeline of the pipeline entry tobe loaded as an input pipeline for an existing search query, or as abasis for a new pipeline that may be created based on the inputpipeline. By selecting use as input link 1106 d, for example, a savedpipeline corresponding to command of the query constructed in searchscreen 1300 of FIG. 13 could be loaded into an interface for creatingand/or modifying a search query that builds on, or extends, the savedpipeline. As shown, the user selecting (e.g., clicking on) use as inputlink 1106 d causes use as input links 1107 a and 1107 b to be displayed.The user selecting (e.g., clicking on) edit pipeline 1108 d causes editpipeline links 1109 a and 1109 b to be displayed. Alternatively any ofthese links may be directly accessible from the pipeline entry, may beaccessible from a different screen and/or interface, or may not beincluded. It is also noted that one or more of the links may not beincluded in a pipeline entry, or may be grayed out depending onpermissions associated with the user attempting to access the savedpipeline. Each pipeline entry in selection interface 1100 can optionallycomprise similar links offering similar functionality as pipeline entry1102 c for the save pipeline corresponding to that entry.

Exemplary functionality of “edit pipeline” links has been describedabove. A primary distinction between edit pipeline 1109 a and editpipeline link 1109 b is the query interface the saved pipeline is loadedinto in response to a user selection of the link. The user can selectedit pipeline link 1109 a, which reads “Edit in Table,” to use the savedpipeline as an input to the search interface corresponding to searchscreen 1200. In response to selecting the link, the system may, forexample, automatically load the saved pipeline into the search interfacecorresponding search screen 1200 and display search screen 1200 to theuser. The user can select edit pipeline link 1109 b, which reads “Editin Search” to use the saved pipeline as an input to the search interfacecorresponding to search screen 600 and display search screen 600 to theuser. In response to selecting the link, the system may, for example,automatically load the saved pipeline into the search interfacecorresponding to search screen 600 and display search screen 600 to theuser.

It is noted that in various implementations, loading a saved pipelineusing an edit pipeline link loads the metadata stored in associationwith the saved pipeline. The metadata can be used to restore the editingsession to its previous state in the search interface. Thus, it shouldbe appreciated that metadata associated with a data object can includeany information necessary to restore a corresponding editing session.FIG. 13 shows search screen 1300 of a search interface, which cancorrespond to search screen 1200. Assume the user constructs a queryusing the search interface and saves the state of the editing session,for example, by selecting save option 1352. The save option can be usedto selectively save the pipeline as a new data object or overwrite aprevious data object.

Saving a new pipeline can cause the pipeline to be displayed inselection interface 1100 of FIG. 11. Selecting edit pipeline link 1109 arestores the state of the editing session to search screen 1300. Thus,the search screen may appear substantially as it did in the table formatwhen the query was initially constructed and form elements of the formsmay include input provided in the previous editing session (e.g., byloading form metadata, table formatting metadata, and/or command entrylist metadata).

It is further noted that selecting edit pipeline 1109 b can similarlyfunction to restore the state of an editing session in a searchinterface corresponding to search screen 600. Furthermore search screen600 may optionally include a save option. Care may be taken to allow foraccessing saved pipelines where an editing session was saved in onesearch interface and editing is resumed in a different search interface.As an alternative, such behavior may be prohibited. In someimplementations, to simplify the handling of metadata, edit pipelinelinks may not be offered for the search interface corresponding tosearch screen 600 (at least for queries constructed in the searchinterface of search screen 1300). Instead, the user may use that searchinterface to create new queries or extend saved queries.

Exemplary functionality of “use as input” links has been describedabove. For example, the user may be unable to modify and/or view theinput query in the search interface, which could depend on permissionsassociated with the user. A primary distinction between use as inputlink 1107 a and use as input link 1107 b is the query interface thesaved pipeline is loaded into in response to a user selection of thelink. The user can select use as input link 1107 a, which reads “Extendin Table,” to use the saved pipeline as an input to the search interfacecorresponding to search screen 1200. In response to selecting the link,the system may, for example, automatically load the saved pipeline intothe search interface corresponding search screen 1200 and display searchscreen 1200 to the user.

FIG. 12A is an example of a search screen resulting from a userselecting an “Extend in Table” link for pipeline entry 1102 d. As shown,in some implementations, the user selecting to load a saved pipelinecauses the previously query constructed to be utilized as an input queryto the current query interface. Exemplary loading of an input searchquery has previously been described above with respect to FIG. 10. Forexample, command entry 1232 a can correspond to command entry 1040 a inFIG. 10. Furthermore, command entry 1232 b may optionally automaticallybe presented to the user to allow the user to remove event attributesfrom query results corresponding to the input query as desired. In theexample shown, the user has selected to remove each unselected eventattribute corresponding to form elements 1209. Initially those eventattributes may have been displayed in the table format. It is noted thatat least some of the metadata of the previously saved editing sessionmay be loaded based upon selection of the extend pipeline link. Forexample, the table formatting metadata can be loaded and applied to thetable format. Thus, table format 1202 may appear substantially as it didto the previous editor of the table format. In some cases, form metadatais excluded from the loaded search interface. Thus user can continue toedit the input query and may save the extended pipeline as a newpipeline.

In FIG. 11, the user can select use as input link 1107 b, which reads“Extend in Search” to use the saved pipeline as an input to the searchinterface corresponding to search screen 600 and display search screen600 to the user. In response to selecting the link, the system may, forexample, automatically load the saved pipeline into the search interfacecorresponding to search screen 600 and display search screen 600 to theuser.

Referring to FIG. 14A, FIG. 14A illustrates search screen 1400 inaccordance with the disclosed embodiments. Search screen 1400 cancorrespond to search screen 600 of FIG. 6A. In response to the userselecting use as input link 1107 b, the saved pipeline (e.g., dataobject) corresponding to pipeline entry 1102 c is loaded into the searchinterface. For example, the user can use search bar 1402 correspondingto search bar 602 in FIG. 6A to extend the previously constructed query.The input query corresponding to the saved pipeline optionally maycorrespond to the input query represented in search screen 1300 of FIG.13, and may have been constricted therein.

As shown, the search bar is optionally automatically loaded with command1404 corresponding to the input query of the saved pipeline. The “from”identifier represents a command that instructs the system to use a savedpipeline having a specified pipeline label as an input query. As shown,the command element “SupplementalData” identifies the saved pipeline.The user can type or otherwise add commands to search bar 1402, such ascommand 1430, to extend the input query. Command 1430 is a “fields”command that removes fields, or attributes, from the query results ifpresent. In the example shown, the user has typed in the command toremove the event attribute having the attribute label architecture fromquery results. The extended query was applied to search screen 1400 toupdate the displayed query results. Further commands could besubsequently added to the query and applied.

In loading the saved pipeline, sidebar 1406 is optionally loaded with atleast some of the same event attributes displayed in the table formatwhen the saved pipeline is loaded into search screen 1300, as shown. Inthe present example, each event attribute displayed in the table formatis loaded into the search interface. The _time and _raw attributes aredisplayed automatically in events list 1408, enabling the user to viewthe raw event data in each of the returned events from the query withcorresponding time stamps. The remaining event attributes are displayedin sidebar 1406. This could be accomplished, for example, by loadingmetadata saved with respect to search screen 1300 that specifies whichevent attributes to display in the table format and loading those eventattributes into search screen 1400 using the metadata.

However, in the present example, this is accomplished by loading eachevent attribute corresponding to the input query into search screen1400. Further, search screen 1300 is configured to display each eventattribute of the query in table format 1302 (e.g., when the most recentcommand entry is selected). Thus, in search screen 1300 the user canremove a column from the table format by removing the event attributeusing the query. It is noted that this could include removing the eventattributes corresponding to _raw and _time. Thus, in someimplementations, the event raw data and/or time stamps can be madeunavailable to the user in search screen 1400. As such, it will beappreciated that at least some of the removing can be accomplished insearch screen 1300 using the query, which is encapsulated by the inputquery (e.g., command 1404) in search screen 1400.

Further, in implementations where the user of search screen 1400 isunable to modify the input query (via permissions or otherwise), theuser may be prevented from viewing transformed states of the input dataprior to the output of the saved pipeline, including removed and/ortransformed event attributes and/or the values thereof. Thus, in somerespects, the data accessible to the user in search screen 1400 can becurated in search screen 1300 for use in search screen 1400. As anexample, sensitive information such as social security numbers and otherpersonal information can be removed from the input data for use insearch screen 1400. It will be appreciated curation of input data can beperformed in either search interface and saved as a pipeline.Furthermore, in each case selecting to extend a saved pipeline mayprevent the user from accessing (viewing or modifying) prior states ofthe input data.

In search screen 1300, command entry 1330 in FIG. 13 corresponds to acommand applied by the user to remove all event attributes except thoselabeled _time, _raw, architecture, client_ip, and salesforce_id. Theuser also interacted with table format 1302 to extract event attributesfrom the event attributes displayed in the table format. An exemplaryoptions menu for performing extractions is shown. The user selectedoptions to generate command entries 1332 and 1334 corresponding tocommands that extract event attributes labeled raw_eloquaGUID(eloquaGUID) and raw_version (version) from an event attribute (theevent raw data in the present example). Thus, as shown in FIG. 14A,those extracted event attributes are available to the user in searchscreen 1400 and loaded into sidebar 1406.

In some implementations, at least some of the event attributes areautomatically selected in sidebar 1406 when loading the saved pipeline.For example, one or more of the fields displayed in the table format canbe loaded as selected fields, as shown. In addition, or instead, one ormore of event attributes could be loaded as an interesting field and bedisplayed as shown in FIG. 6A. It is noted that the event attributehaving the attribute label architecture may have initially been aselected field (or interesting field) in search screen 1400, but wasremoved when the user removed that event attribute from the queryresults using command 1430. It is noted that user may optionally addfilters to the query that filter events from query results based onspecified values of event attributes (e.g., all returned events mustinclude a specified value(s) for a specified event attribute or allreturned events must not include a specified value(s) for a specifiedevent attribute).

In search screen 1400, the user can select and deselect any of the eventattributes displayed in sidebar 1406. By selecting an event attribute(e.g., an interesting field), the event attribute becomes a selectedfield. A user can click on any event attribute in sidebar 1406 to causedisplay of a form similar to form 1440, which is customized to thechosen event attribute. Form element 1432 can be used to toggle betweena field being an interesting field and a selected field. In someimplementations, one or more of the interesting fields may optionally beselected and/or generated by the system by extracting the interestingfields from the event raw data. These extractions may be preconfiguredby a user and/or based on one or more default extraction rules, such asthe extraction rules of an autoextract command described above. In somecases, these interesting fields may not be displayed where an inputquery is loaded into the search interface, may not automatically bedisplayed, or may be distinguished from interesting fields defined bythe input query. For example, in the present example, the searchinterface only displays interesting fields that are defined by the inputquery when the query is initially loaded. A user may later generateother interesting fields from the event raw data if the data isavailable in the query results. Also shown in FIG. 14A, form 1440displays statistics about the chosen event attribute and can be used togenerate various reports on the event attribute.

Also shown in search screen 400, selected fields are displayed in eventslist 1408. In particular, values of the fields are displayed with theraw data that corresponds to the same event. In some implementations,selected fields are displayed in events list 1408 and the user canremove fields from events list 1408 by unselecting the field (e.g.,using form element 1432) or removing the field using the query (e.g.,using search bar 1402). Events list 1408 offers some interactivity suchas allowing the user to select keywords in the raw data to add to thequery.

Search screen 1400 also optionally includes a load in table option 1442.The load in table option is displayed as a button, and is selectable bythe user to load the query (and optionally various metadata such asselected fields, interesting fields, and/or user generated extractionrules for fields) corresponding to search bar 1402 in the searchinterface corresponding to FIG. 12. Exemplary operation of the option isdescribed in further detail with respect to FIG. 14B.

3.8 Transitioning Between Query Interfaces

Referring to FIG. 14B, FIG. 14B illustrates search screen 1400 inaccordance with the disclosed embodiments. In FIG. 14B, search bar 1402corresponds to a new query. The user may have deleted commands 1404 and1430 in FIG. 14A and typed in a new query, for example, such that theinput query is no longer utilized. As another example, the user may haveselected option 1435 causing search screen 1400 to be loaded without theinput query. The user may have subsequently typed in the new query. Asshown, in some implementations, selecting to create a new query causesdisplay of interesting fields include at least some fields that do notcorrespond to event attributes of query results. As described above,these interesting fields can be based on user and/or system definedextraction rules that may be preconfigured in the system.

In search bar 1402, the user has specified the sourcetype of the queryas “product_downloads” to filter events having other sourcetypes fromquery results. Further, the user has selected various fields shown insidebar 1406 and potentially unselected other fields. Additionally, theuser has constructed the query to filter events where the eventattribute labeled client_ip has an undefined value.

After performing initial curation of the input data via the query, theuser may desire to load the query into the search interfacecorresponding to search screen 1200. In some implementations, this canbe accomplished by clicking, or otherwise selecting load in table option1442. Referring to FIG. 15, FIG. 15 illustrates search screen 1500 inaccordance with the disclosed embodiments. Search screen 1500corresponds to the query of search screen 1400 in FIG. 14B loaded intoan interactive table format, which can correspond to the result of theselecting of load in table option 1442 by the user.

As shown, in some implementations, the user selecting to transition tothe search interface corresponding to search screen 1500 causes thequery constructed in the previous query interface to be utilized as aninput query to the current query interface. Exemplary loading of aninput search query has previously been described above with respect toFIG. 10. For example, command entry 1540 a can correspond to commandentry 1040 a in FIG. 10. As an alternative, each portion of the querydivided by a pipe symbol “1” could have been loaded as a respectivecommand entry in command entry list 1508.

As shown, in some implementations, the user selecting to transition tothe search interface corresponding to search screen 1500 causes one ormore event attributes of query results of the query to be automaticallydisplayed in table format 1502. The displayed event attributes may bedetermined by the user in the previous query interface. For example, theevent attributes displayed in table format 1502 are based on theselected fields in search screen 1400. In the present implementation,each selected field is automatically displayed in the table format.Optionally, as shown, the event attributes labeled _time and _raw arealso automatically displayed in the table format, if present. Thus, theuser can continue to construct the query based on progress made usingthe previous search interface. The selected fields may optionally beextracted from metadata associated with the previous query interface andapplied to the metadata associated with the new query interface.

In some implementations, one or more event attributes are automaticallyremoved from the query results in transitioning to search screen 1500.For example, one or more commands may be automatically added to thequery to remove the one or more event attributes. Furthermore, one ormore command entries can be automatically added to command list 1508that correspond to the one or more commands. For example, FIG. 15 showscommand entry 1540 b corresponding to a command that causes the query toremove designated event attributes from query results. In the presentimplementation, selected fields are retained from the previous queryinterface and other fields are automatically removed from the queryresults using the command. In other implementations, the one or morecommands could be encapsulated in command entry 1540 a.

In some implementations, each field in the query results is displayed intable format 1502. Thus, by automatically removing some of the fields,those fields are not displayed in table format 1502. As the user may notbe interested in unselected fields from the previous query interface, itcan be inferred that those fields should be excluded from table format1502.

In some implementations, the user can selectively add event attributesto table format 1502 that were automatically excluded from table format1502. For example, command entry 1540 b can be selected by the user tocause display of a form comprising event attributes corresponding to theinput query. As shown, the form includes form elements 1509 and isdisplayed in panel 1505 corresponding to interface panel 1205. The eventattributes displayed in the form include, by way of example, interestingfields, selected fields, an event attribute corresponding to event rawdata (i.e., _raw), and an event attribute correspond to event timestamps (i.e., _time). The user can interact with the form elements tounselect and select event attributes as described to control the eventattributes of the query and the display of event attributes in tableformat 1502. In the present example, each event attribute corresponds toa form element and the user selects or unselects the event attribute byclicking on the form element. The users selections can be applied totable format 1502, for example, by clicking apply button 1550.

As shown, the users can continue to modify the query through interactionwith table format 1502. Further, the user can selectively save thepipeline, for example, by selecting save option 1552. As in each searchscreen, when available the save option can be used to selectively savethe pipeline as a new data object and/or overwrite a previous dataobject. Saving a new pipeline can cause the pipeline to be displayed inselection interface 1100 of FIG. 11.

3.9 Runtime Permissions of Queries

In further respects, the present application relates to assigningruntime permissions to queries saved in association with query objects.Examples of query objects include data objects, such as saved pipelinesdescribed above. Runtime permissions that can be assigned to a queryinclude access permissions to query one or more data sources. The accesspermissions grant the query access rights to input data for the queryfrom the one or more data sources. By allowing users to execute dataqueries using the runtime permission assigned to the query, users thatotherwise may not have access rights to at least some of the datasources can receive the same query results. Thus, it can be ensured thatthe query results are consistent amongst different users that may havedifferent permissions.

For example, assume the input data provided to the query correspondingto search screen 1200 in FIG. 12A is from a first data source and asecond data source. Each data source may have at least some extractableevent attributes in event raw data that is unique to that data source.For example, the event attribute corresponding to activity may only beextractable from the first data source. Further assume that a first userhas access permissions to the first and second data sources and a seconduser only has access permissions to the second data source. Applying theuser permissions to the query results in table format 1202 includingcolumn 1204 b and the associated events for the first user, but not thesecond user. However, it may be desirable to ensure that both usersreceive the same query results.

One approach to achieving consistent query results amongst users is tochange the access permissions of the second user to allow access to bothdata sources. However, the first data source may contain sensitiveinformation that should not be made accessible to the second user forsecurity or privacy reasons. By allowing the second user to execute thedata query using the runtime permission assigned to the query, thesecond user can access the same query results as the first user, whilethe sensitive information can be removed by the query. Thus, the seconduser's level of access to the second data source can effectively be madegranular and specific rather than applying a blanket policy.

In further respects, the access permissions that are assigned as theruntime permissions can be of users. For example, the access permissionsmay be of a creator of the query (e.g., the initial creator of thequery). In various implementations, the creator of the query creates thequery in a query interface that allows the creator to view query resultscorresponding to the query. Further the query interface can allow thecreator to update the query and see updated results of the query.Examples of such query interfaces are described throughout the presentapplication and correspond to at least search screens 600, 800, 1200,1300, 1400, and 1500.

In some implementations, the access permissions are assigned as aruntime permission of the query based on the user updating a queryobject and/or creating a new query object. For example, in response tothe user selecting an option to save a query in association with a queryobject, the user's access permissions may be stored as runtimepermissions in association with the query object and the saved query.The option could correspond to save option 1352 as one example. Bysaving the user's access permissions as a runtime permission of thequery, the user can be ensure that subsequent users that execute thequery can access query results that are consistent with the queryresults used to create the query.

Certain users can be denied access to previous state of input data bypreventing the users from executing a modified version of the query(e.g., using permissions and/or the interface design). As an example,some users may be denied access to any edit pipeline links for the query(e.g., based on permissions). Typically the creator of the query and/oradministrators will have access permissions for editing the query (e.g.,automatically by default). Furthermore, in some cases administrators mayselectively grant or remove these permissions for users on an individualor group basis. Additionally, the creator and/or administrators mayselectively grant or remove the permissions to access the query forexecution). Typically the creator of the query and/or administratorswill have execution permissions for the query (e.g., automatically bydefault). In some cases administrators and/or the creator mayselectively grant or remove these permissions for users on an individualor group basis.

In various implementations, as long as a user has execution permissionsfor the query, the user is allowed to execute the query with the runtimepermissions (e.g., regardless of the user). However, it is noted that insome implementations, the runtime permissions may be designated for oneor more particular projects and the allowing the second user to executethe search using the determined runtime permission is for the particularproject(s). As an example, the runtime permissions may be designated bythe creator, an administrator, and/or automatically by the system (e.g.,where the project used to create the query is designated). In addition,or instead, runtime permissions may be designated for one or moreparticular apps in the project environment. “Apps” as used herein mayrepresent software programs and processes that may implement variousfeatures and processing. Apps may give users insights into theirprojects via dashboards, reports, data inputs, and search sessions(e.g., in a search interface) that work in the project environment inwhich they are installed. Apps may completely reconfigure the way aproject interface looks or may launch in an entirely separate interface.

In some respects, a query can be executed as a base query and/or one ormore subqueries to a base query. A base query corresponds to a completedata processing pipeline. A base query can optionally comprise one ormore subqueries. A subquery corresponds to data processing pipeline thatprovides the output query results of the data processing pipeline to adata processing pipeline of a base query. In some implementations, asubquery can serve as a base query to one or more other queries. Thedata processing pipeline of a base query supplements its query resultswith the query results of a subquery. The supplemented query results mayfurther be processed by the data processing pipeline to provide queryresults as an output data set of the base query.

Referring to FIG. 16, FIG. 16 shows diagram 1600 depicting exemplaryexecution of primary query 1602 in accordance the disclosed embodiments.A primary query corresponds to a base query that provides final queryresults of a data query executed by the system as an output data set. Adata query typically comprises a single primary query. As shown in FIG.16, primary query 1602 receives input data from data source 1650 andprovides output data set 1652 comprising query results of the dataquery. Primary query 1602 corresponds to a data processing pipeline andmay be in a pipelined query language, such as SPL or a data processingscript. The execution of primary query comprises execution of multiplesubqueries 1604, 1606, 1608, 1610, and 1612.

In various implementations, a primary query can comprise any number ofsubqueries. An input query may refer to a subquery that serves as aninput data source to another query. A supplemental query may refer to asubquery that service as a supplemental data source to another query. Asshown, subqueries 1604 and 1606 each receive input data from data source1654 and serve as supplemental data sources to primary query 1602 atdifferent points in the data processing pipeline. In particular queryresults of those subqueries are provided to the data processing pipelineof primary query 1602. Similarly, subquery 1608 receives input data fromdata source 1656 and serves as supplemental data source to primary query1602 at a later point in the data processing pipeline of primary query1602 (e.g., in a later command) Also in FIG. 16, subquery 1612 receivesinput data from data source 1658 and serves as an input data source ofsubquery 1610. In turn, subquery 1610 processes input data from subquery1612 and serves as supplemental data source to subquery 1606 at a pointin the data processing pipeline of subquery 1606.

In some cases, a subquery may be incorporated into a base queryutilizing one or more commands of the base query (e.g., using a queryidentifier of the subquery in the commands). For example, a supplementalquery may be referenced by in a lookup, join, or other data combinationcommand in the base query. Exemplary combination types have beendescribed above with respect to FIGS. 12E, 12F, and 12G. Command entry1234 in FIG. 12G represents a command that references a query as asupplemental data source. As an example of a reference to a query as aninput data source, command entry 1232 a represents a command thatreferences a query as an input data source. It is noted that commands inthe query need not be required to reference a query as an input datasource or supplemental data source. For example, a reference may be madeusing another approach, such as via metadata associated with a queryobject of the base query or by otherwise storing the association.

From FIG. 16 is can be seen that execution of a primary query may becomplex, including numerous queries serving as input data sources andsupplemental data source of each other and/or the primary query. Inexecuting the primary query, were users' access permissions applied,different users may receive vastly different versions of output data set1652. Additionally, the various queries depicted in FIG. 16 may becreated by many different users having many different accesspermissions. Were the executing of the primary query to utilize theruntime permissions of the primary query for each of the subqueries, thefinal query results may include more information than is desirable fromthe user executing the primary query. Thus, in some implementations,each subquery of a primary query is executed using the runtimepermissions associated with the subquery (e.g., of the creator of thequery). As with primary queries, each subquery may correspond to arespective query object. For example, each subquery may correspond to arespective saved pipeline having a corresponding pipeline entry inselection interface 1100 of FIG. 11. It should be appreciated that aquery may be executed as a subquery of another query, or may be executedas a primary query itself. The saved pipeline corresponding to pipelineentry 1102 c is one such example.

In various implementations, the data sources corresponding to theruntime permissions are data stores. For example, the data stores can beindexed data stores and the query can be on indexed items in in the datastores. In some implementations the indexed items comprise indexedunstructured data, such as event raw data. Referring to FIGS. 18A and18B, FIGS. 18A and 18B show exemplary permissions interface 1800 formodifying and assigning access permissions of users, such as those thatmay be assigned as runtime permissions of saved pipelines. Someimplementations employ roles, such as those shown in FIG. 18A fordefining the access permissions. Roles represent definitions of accesspermissions and authorizations that enable users to execute variousfeatures, apps, or access data, in the various embodiments. Roles maydefine how user may interact with apps, indexes, indexers, and the like.For example, a user may be assigned one or more roles. The roles shownin FIG. 18A include admin role 1802 and user role 1804. Any number ofother roles can be employed.

In some implementations, a user selects a role and can perform one ormore modifications any of the various aspects of the role, such asaccess permissions and authorizations. As an example, FIG. 18B may bethe result of a user accessing user role 1804 for this purpose from FIG.18A. For example, the user may be assigned admin role 1802 and click onthe link corresponding to user role 1804 to access the screen shown inFIG. 18B. As shown in FIG. 18B, the user can alter the accesspermissions of the user role by selecting from the list of data stores1810 which data stores are to be restricted for the user role. Theseaccess permissions may be the access permissions assigned to savedpipelines as runtime permissions thereof.

Referring to FIG. 11, a user may edit runtime permissions of queries,for example, by selecting a corresponding link of a saved pipelineentry. As an example, a user may edit runtime permissions of the savedpipeline corresponding to pipeline entry 1102 c by selecting editpermissions link 120. FIG. 17 shows exemplary permissions form 1700,which may be presented to the user in response to selecting editpermissions link 1120. As shown, by default, the saved pipeline may beconfigured to run using the access permissions corresponding to theowner (e.g., the creator), as indicated by the selection of form element1706. However, the user may optionally change the runtime permissions tocorrespond to another user. As an example, the runtime permissions maybe changed to correspond to the user executing the query by selectingform element 1708.

Permissions form 1700 further shows that the user may optionally be ableto determine Read and Write permissions for the saved pipeline withrespect to other users, such as via selection of form elements 1710 forroles. In some implementations, Write permissions define whether a usercan edit the saved pipeline (e.g., via an edit pipeline link of FIG.11). Furthermore, Read permissions may define whether a user can extendthe saved pipeline (e.g., via an extend pipeline link of FIG. 11). Theuser may subsequently apply any changes the user was permitted to maketo the permissions in permissions form 1700.

4.0 Illustrative Search Screen with Summary View Functionality

FIG. 19A depicts an illustrative search screen 1900 in accordance withvarious implementations of the present disclosure. As with searchscreens 800, 1200, 1400, and 1500, discussed above, search screen 1900may be utilized as part of a search interface to display one or moreevents of a set of events that satisfy a search query (e.g., searchquery 1914 represented in a command entry list). As used herein, tosatisfy search query 1914, the set of events would be those events thatconform with search query 1914. Put another way, to satisfy search query1914 the set of events would be returned by the execution of searchquery 1914.

Search screen 1900 may display a subset of the set of events in a tableformat 1902. Table format 1902 may be the same as, or similar to, thevarious table formats discussed above. As such, table format 1902 mayinclude a plurality of rows 1906 a-1906 g. Each of rows 1906 a-1906 gcan represent an event of the set of events. In addition, table format1902 may include a plurality of columns 1904 a-1904 f Each column of theplurality of columns can represent a respective event attribute. Forexample, columns 1904 a-1904 f represent event attributes _time, date,AccountName, AccountRegion, AccountSubRegion, and AccountTheater,respectively. It will be appreciated by those of skill in the art thatthe event attributes depicted are selected for purposes of example onlyand should not be viewed as limiting of this disclosure. The eventattributes represented by columns 1904 a-1904 f can be a subset of theevent attributes contained within the set of events or the entirety ofthe event attributes contained within the set of events. The columns oftable format 1902 form cells (e.g., cell 1912) with the plurality ofrows of table format 1902. As can be seen, these cells are formed wherethe columns of table format 1902 intersect with the plurality of rows oftable format 1902. Each individual cell may include data items, orvalues of data items, of the respective event, represented by the row inwhich the individual cell lies, and the respective event attribute,represented by the column in which the individual cell lies.

Search screen 1900 may be utilized as part of a search interface(corresponding to the search interface of search screens 800, 1200, andthe like) that allows a user to modify search query 1914. Someillustrative options for modifying search query 1914 include anycombination of deleting commands from search query 1914, adding commandsto search query 1914, reordering one or more commands in search query1914, and modifying variables, parameters, arguments, and/or otherproperties of commands in search query 1914, as has been previouslydiscussed herein.

Search screen 1900 may also be utilized to update the search result setto correspond to the modified search query and to update the events thatare displayed in search screen 1900 to correspond to the updated searchresult set. In some cases, based on a search query being modified, thesearch query could be completely re-executed to retrieve new searchresults and generate the updated search result set, which could then bedisplayed in table format 1902. In other cases, the search query mayonly be partially executed. For example, in implementations where apipelined query language, such as SPL, is employed for search queries,additional commands that are added to a search query may be applied toat least some previous search results. These and other variations arepossible for updating the search result set to correspond to a searchquery.

By interacting with search screen 1900 to create and/or modify searchqueries, a user may utilize the search interface to filter, sort, clean,enrich, analyze, report on, and/or otherwise carryout functionalityprovided for by commands in search queries. Furthermore, as the usergenerates modified search queries, the search result set can be updated,with events displayed in search screen 1900 (e.g., in table format 1902)being updated to reflect the modifications. Utilizing this approach, auser may iteratively modify a search query and view the impact of themodification via updated search results. This approach can be employedto enable users to effectively and efficiently generate queries thatreturn expected and desired results, even without extensive knowledge ofthe underlying commands and/or search language employed by the queries.

In embodiments, search screen 1900 may be configured by the system toenable a user to switch to a data summary view in which the user canview summary reports for event attributes represented by one or more ofcolumns 1904 a-1904 f of table format 1902. Such a configuration isdepicted by control 1910 which can be a button configured to cause thedata summary view to display upon selection by a user. An illustrativedata summary view is depicted in FIG. 19B.

4.1 Illustrative Data Summary View

FIG. 19B depicts an illustrative data summary view 1946 that the systemcan cause to be presented. Data summary view 1946 is configured todisplay summary reports for the event attributes represented by each ofone or more of columns 1904 a-1904 f. As depicted in FIG. 19B, a summaryreport is presented for each of the event attributes represented bycolumns 1904 a-1904 f, however, as discussed further in reference toFIG. 19C, in other embodiments, any subset of the event attributesrepresented by columns 1904 a-1904 f may be presented by summaryreports. Data summary view 1946 may be presented, in someimplementations, as an overlay over the above discussed table format1902. A user can utilize control 1920 to selectively return to the abovediscussed table format 1902. As such, the system can enable a user toswitch from table format 1902 to data summary view 1946 via control 1910from within table format 1902 and to switch from data summary view 1946back to table format 1902 via control 1920 from within the data summaryview.

As depicted, each summary report may include a coverage indicator(s)(e.g., coverage indicators 1948 a-1948 f), a summary graph(s) (e.g.,summary graphs 1922 a, 1922 b, and 1922 f), and/or summary entries(e.g., summary entries 1924 b and 1924 f) for the respective eventattribute which the summary report is summarizing. The summary reportsdepicted may present summary information for the set of events returnedby search query 1914 while table format 1902 may not necessarily depictall of the events at any one time. As such, the summary report maypresent the user with a bigger picture of what is occurring in the datathan the table format is capable of presenting. It will be appreciatedthat these components of the summary report are merely meant to beillustrative of possible components of the summary report and that anypresentation of data summarization is within the scope of thisdisclosure. In addition, as discussed in greater detail below, theinformation displayed in the summary report, or the format of thesummary report, may be based on a data type associated with therespective event attribute of the summary report.

The coverage indicator can indicate the percentage of events, of the setof events, that contain a data item for the respective event attribute.For example, the coverage indicator of event attribute _time indicatesthat there is 100% coverage for this event attribute. As such, eachevent of the set of events includes a data item for the _time eventattribute. The coverage indicator for AccountSubRegion, on the otherhand, indicates that there is only 25% coverage for this eventattribute. As such, only 25% of the events in the set of events includea value for AccountSubRegion. The coverage indicator may be beneficialto enable a user to determine the percentage of events, of the set ofevents, that are included within the summary report. For example, thecoverage indicator might be indicative of the relative importance of theevent attribute associated with the coverage indicator or could indicateto the user that the query is returning unexpected or unwanted resultsand thus the query may need to be amended.

The summary entries are depicted in the area located below the summarygraphs. Because the summary graphs, as illustrated, depict at least asubset of the summary entries within a summary report, the summaryentries will be described first. In embodiments, the summary entries mayinclude a selection of values that occur within the data items of therespective event attribute. In addition, summary entries may includesummary statistics, in some embodiments. As used herein, summarystatistics refer to results of any statistical, or mathematical,analysis of the set events (e.g., summary entries 1924 b). In someembodiments, each summary entry may include a text component thatcorresponds with, or is based on, values of the data items of therespective event attribute. In addition, the summary entry may include aportion of a key, or legend, for the summary graph that identifies thesection of the summary graph that corresponds with the summary entry.The summary graphs are discussed in greater detail below, however, itwill be appreciated that not all summary entries need be represented inthe summary graph.

In some implementations, the values of at least a portion of the summaryentries may represent top values. These top values may identify one ormore values within the data items of the respective event attributewhose occurrences exceed an upper occurrence threshold. In someembodiments, this upper occurrence threshold may be a user defined upperoccurrence threshold. For example, if the user is only interested insummarizing those values of the data items of the respective eventattribute that occur in at least 10% of the set of events, the usercould designate 10% as the upper occurrence threshold. As anotherexample, the user could be interested in only a specific number ofvalues that occur the most within the data items of the respective eventattribute, regardless of percentage. In such an example, the user coulddesignate a set number (e.g., 5) of values to display as top values. Assuch, the system could be configured to select the top 5 values based onoccurrence, which would effectively make the lowest percentageoccurrence of these top 5 values the upper occurrence threshold. Inother embodiments, the upper occurrence threshold may be definedprogrammatically. For instance, the upper occurrence threshold couldvary based on the data items of the event attributes. For example, thesystem could be configured to select only those values that occur asufficient number of times to make the values statistically significant.It will be appreciated that these examples are merely meant to beillustrative of possible mechanisms for selecting the top values andthat any other suitable mechanism could be utilized without departingfrom the scope of this disclosure. In some embodiments, the top valuesmay also include a catch-all category, such as the ‘others’ categorydepicted in top values 1924 f. This catch-all category may represent allof those values whose occurrences did not exceed the upper occurrencethreshold. This catch-all category may enable a user to visualize thepercentage of values that did not exceed the upper occurrence thresholdvia the summary graphs (or may only collectively exceed the upperoccurrence threshold).

In addition to the top values discussed above, in some embodiments thesummary entries may include a selection of rare values (e.g., rarevalues 1928). These rare values may, in some embodiments, identify oneor more values within the data items of the respective event attributethat occur below a lower occurrence threshold. In some embodiments, thislower occurrence threshold may be a user defined lower occurrencethreshold. For example, if the user is only interested in summarizingthose values of the data items of the respective event attribute thatoccur in less than 1% of the set of events, the user could designate 1%as the lower occurrence threshold. As another example, the user could beinterested in only a specific number of values that occur the leastwithin the data items of the respective event attribute, regardless ofpercentage. In such an example, the user could designate a set number(e.g., 4) of values to display as rare values. As such, the system couldbe configured to select the lowest occurring 4 values which wouldeffectively make the highest percentage occurrence of these 4 values thelower occurrence threshold. In other embodiments, the lower occurrencethreshold may be defined programmatically. For instance, the loweroccurrence threshold could vary based on the data items of the eventattributes. For example, the system could be configured to select onlythose values that occur a sufficiently low number of times to make thevalues statistically insignificant. It will be appreciated that theseexamples are merely meant to be illustrative of possible mechanisms forselecting the rare values and that any other suitable mechanism could beutilized without departing from the scope of this disclosure. In otherembodiments, the system could be trained on what constitutes a rarevalue (e.g., by a user designating a specific value as rare ordesignating a value selected as rare to not be rare). Such trainingcould be accomplished via interaction with either the table formatdiscussed herein or summary elements of the data summary view.

In addition to the top values and rare values discussed above, in someembodiments the summary entries may include a selection of unusualvalues (e.g., unusual values 1926). These unusual values may identifythe values of one or more data items of the respective event attributethat are beyond a threshold of similarity from the other data items ofthe respective event attribute. The similarity between one data item andanother data item can be determined in any conventional manner fordetermining similarity between two items. In addition, the similaritybetween two data items can be determined based on any aspect of the dataitems, such as, for example, a type (e.g., data type) associated withthe data item, a value contained within the data item, etc. As with rarevalues, in other embodiments, the system could be trained on whatconstitutes an unusual value (e.g., by a user designating a specificvalue as unusual or designating a value selected as unusual to benormal). Such training could be accomplished via interaction with eitherthe table format discussed herein or summary elements of the datasummary view.

The summary graphs (e.g., summary graphs 1922 a, 1922 b, and 1922 f)depict a distribution of at least a subset of values of the data items,of the represented event attribute over time. As depicted, the summarygraphs illustrate the distribution of values of the data items that areincluded within the summary entries discussed above. For example,summary graph 1922 f depicts in sections 1932 a-1932 d the distributionof top values 1930 a-1930 d, respectively, over time. It should be notedthat top value 1930 d represents the previously discussed catch-allcategory. As such, section 32 d of summary graph 1922 f enables a userto visualize the percentage of values that did not exceed the upperoccurrence threshold. As depicted, summary graph 1922 f depicts adistribution for 100% of the values of the data items of theAccountTheater event attribute in the event set. Put another way, thecombination of the top values when including the catch-all category of‘others’ represent 100% of the possible values for the AccountTheaterevent attribute in the event set. It will be appreciated, that this ismerely for illustrative purposes only and should not be viewed aslimiting of this disclosure. Any other depictions that enablevisualization of the values of the data items of the respective eventattribute are explicitly contemplated.

As mentioned previously, the format or content of the summary report mayvary automatically depending on a data type of the respective eventattribute. The data type may be, for example, a time data type, anumerical data type, a categorical data type, or a user defined datatype. It will be appreciated that these data types are selected forpurposes of illustration only and that any other conventional data typeis within the scope of this disclosure. As can be seen, the summaryreport of column 1904 a is depicted as a time data type represented bysymbol 1950 a, the summary report of column 1904 b is depicted as anumerical data type represented by symbol 1950 b, and the summary reportof column 1904 c is depicted as a categorical data type represented bysymbol 1950 c.

As can be seen, the summary report of the time data type represented incolumn 1904 for the _time event attribute does not depict any summaryentries, but merely summary graph 1922 a that depicts a count of thedata items over time. There is 100% coverage for the _time eventattribute and this graph depicts a count of the events of the set ofevents over time.

The summary report of the numerical data type represented in column 1904b, on the other hand, depicts a distribution of values of the data itemsof the date event attribute over time. The values have undergone anumerical, or statistical, analysis to identify a maximum value, aminimum value, a standard deviation, an average value, a mode value, anda median value (not depicted) of the values of the data items of thedate event attribute. As used in this context, average value refers tothe mean value of the values of the data items. In addition, as used inthis context, mean, median, and mode are to be interpreted inconformance with their respective understanding in conventionalnumerical analysis. It will be appreciated that this numerical analysisis merely meant to be illustrative of possible numerical analysis. Anyother numerical analysis, statistical analysis, etc. that could enable auser to visualize the values of the data items of the respective eventattribute are explicitly contemplated herein. It will also beappreciated that these values of the date event attribute represent anumerical conversion of the dates presented in the table format depictedin FIG. 19A. As can be seen in summary graph 1922 b, the maximum,minimum, standard deviation, and average are all depicted in the graphto enable a user to visualize the occurrences of these values. It willbe appreciated that the statistical analysis discussed above extendsbeyond merely a numerical data type and could be applied to any type ofdata. For example, this statistical analysis could include determiningcardinality of various sets of values of the data items of therespective event attribute. As such, any statistical analysis that couldenable a user to visualize the values of the data items of therespective event attribute are explicitly contemplated herein regardlessof a data type associated with the values.

The summary report of the categorical data type represented in column1904 c, in contrast, depicts a distribution of values of the data itemsof the AccountName event attribute over time without the above discussednumerical analysis. As discussed above in reference to summary graph1922 f, summary graph 1922 c depicts sections of the graph thatcorrespond with the top values presented in the summary entriesrepresented over a period of time. As can be deduced, and is discussedabove, the combination of the top values when including the catch-allcategory of ‘others’ represent 100% of the possible values for theAccountName event attribute in the event set. It will be appreciated,that this is merely for illustrative purposes only and should not beviewed as limiting of this disclosure. Any other depictions that enablevisualization of the values of the data items of the respective eventattribute are explicitly contemplated.

4.2 Illustrative Interactions with Data Summary View

In some implementations, a user can interact with the summary entriesand/or the summary graphs depicted in the summary reports. Such aninteraction may enable display of additional information to the userconcerning a selected summary entry or a selected portion of a summarygraph; enable the user to revise the search query with respect to aselected summary entry or a selected portion of a summary graph; orenable filtering of the set of events with respect to a selected summaryentry or a selected portion of a summary graph (e.g., to allow the userto perform a faceted analysis of the events in the set of events). Theseinteractions are discussed in greater detail below.

As mentioned above, in some implementations the system may enableinteraction with summary entries and/or the summary graph to causedisplay of additional information for a selected summary entry or aselected portion of the summary graph. As an example, selection of thecatch-all category ‘others,’ either in a summary entry or a section of asummary graph, could display the values of the data items of therespective event attribute that are included within this catch-allcategory. As another example, selection of a value that is designated asa rare value in the summary entries may identify a percentage or numberof occurrences that are attributed to that value. As an even furtherexample, selection of a value that is designated as an unusual value inthe summary entries may identify what aspects of the value, or the dataitem, make the value unusual (e.g., data type, format, etc.). Such aselection may be accomplished via a mouse-over event, a left click, aright click, a touch, or any other suitable form of interaction with asummary entry, or a corresponding portion of a summary graph.

In addition to displaying additional information, the system can beconfigured to cause interaction with a summary entry and/or a summarygraph to enable the user to revise the search query with respect to aselected summary entry or the selected portion of a summary graph. Toaccomplish this, the system can be configured to enable a user to make aselection of a portion of a summary graph (e.g., a value or range ofvalues) or a summary entry of a respective summary report. Thisselection may be accomplished through any conventional manner,including, but not limited to, mouse-over, left-click, right-click,touch selection, etc. Based on the selection, the system can beconfigured to cause one or more options to be displayed (e.g., list ofoptions 1918) corresponding to the selection. Based on the userselecting one of the displayed options, commands, based at least in parton the selected option, can be added to search query 1914.

The one or more commands that are added to the search query (e.g., tothe end of the search query) can potentially be based on additionalfactors, such as one or more data items and/or one or more eventattributes of the selection. Furthermore, a corresponding command entrymay be automatically added to the command entry list if present. In somecases, the updated query is automatically executed and the eventsdisplayed in search screen 1900 are updated to correspond to the updatedquery. Where the data summary view is still displayed, the data summaryview may be updated to reflect the updated events. In someimplementations, the view is automatically returned to table format 1902instead of data summary view 1946 and the table format is updated toreflect the updated query.

Examples of selectable portions of a summary report include any portionof a summary graph of the summary report, including a value or range ofvalues within the summary graph, or summary entries contained within thesummary report. As an example, in some implementations, one or moresummary entries and or portions of a summary graph may be selectablewithout necessarily requiring other summary entries and/or otherportions of the summary graph to be selectable. The same is true forother types of summary report elements, such as, for example, columns.

Although many approaches exist for selection of a portion of a summarygraph and/or a summary entry, hereinafter referred to collectively asselectable portion(s) for simplicity, in some implementations, aselectable portion(s) may be highlighted or otherwise emphasized when apointer that is displayed in the user interface moves over a particularregion of the display (e.g., a region of the summary report) thatcorresponds to the selectable portion(s). This feature is also referredto as highlight with rollover (e.g., detected when a pointer moves overa region, or upon occurrence of a mouse-over event). One or morehighlighted selectable portions can then be selected in response toadditional user input, such as a mouse click or touch input to selectthe selectable portions. A shift-click or other method could be utilizedto select additional selectable portions or to select a range within asummary graph.

By way of example, in search screen 1900 of FIG. 19A, each summary entrycan be individually selectable and each section of a summary graph canbe individually selectable. The region for each of these selectableportions is substantially coextensive with the portion of the searchscreen 1900 covered by the selectable portion, as well as possibly theimmediate area surrounding the selectable portion. For example, theregion for selectable portion 1916 can include the text “West,” as wellas the block next to the text that acts as key, or legend, for thesummary graph, and any space in between. Furthermore, each column isindividually selectable and the region for each column is coextensivewith the column's header, which comprises an attribute label of theevent attribute of the column.

4.3 Illustrative Options within the Data Summary View

A variety of approaches are available for presenting options that aredisplayed based on and corresponding to the selection of one or more ofthe above discussed selectable portion(s) of summary reports of the datasummary view. In some implementations, options can be presented as alist of selectable options. Options may appear as an option menu (e.g.,option menu 1918), or elsewhere. Display locations of option menus canbe based on the location of the selectable portion(s) that are selectedby the user. For example, option menus can be configured to appearproximate to (e.g., over, or adjacent to) one or more selected portions,as illustrated by option menu 1918.

In the present implementation, each option of the option menu cancorrespond to one or more commands that may be included in search query1914. However, in some cases, options need not correspond to one or morecommands that may be included in a query. Instead, the option may beoperable to interact with the system in some other manner. Where, anoption corresponds to a command, the command may be provided to a searchquery utilizing a format that includes a command identifier thatidentifies the command and one or more command elements of the command,at least some of which may be optional (e.g., arguments, parameters,values, command options, and the like). In particular, each commandcould correspond to a pipelined search language command, such as an SPLcommand, or another type of command compatible with processing of thequery.

In various implementations, option menu 1918 can be context based. Inthis regard, one or more of the options in option menu 1918 can beincluded based on a context related to selected portion 1916. Forexample, option menu 1918 may include options that correlate with a datatype of the respective event attribute associated with selected portion1916. The data type of the AccountSubRegion event attribute representedby column 1904 e is a categorical data type and therefore options mayinclude modifying the search query so that the set events only depictthose events whose AccountSubRegion event attribute is equal to ‘West’or those events whose AccountSubRegion event attribute is not equal to‘West.’ If it were a numerical data type, on the other hand, a differentarray of options may be presented, including relational operators suchas, for example, >, <, =, ≥, ≤, ≠ that may be utilized to add one ormore commands to the search query. As an example, if a user were toselect the average value depicted in column 1904 b then the user couldbe displayed a list of options that include modifying the search queryso that the set of events only depicts those events that satisfy one ofthe relational operators listed with respect to the selected averagevalue. These relational operators may not make much sense and could beconfusing if included in an option menu for a non-numerical data type.

Context can be further based on a source of data items in the selectedportion(s) of the data summary view. For example, one or more optionsmay be included in or excluded from an option menu based on adetermination by the system that at least one of one or more data itemsof at least one of the selected portion(s) comprises a statistical valuegenerated by one or more statistical functions performed on values ofdata items of at least some event attributes. A statistical value mayrefer to a value generated from an event using one or more statisticalfunctions (e.g., average, sum, mean, median, mode, standard deviation,variance, count, range), such that the value no longer correspondsdirectly to event raw data. In some cases, a value may be determined asa statistical value based on identifying the value as an output of astatistical command in a search query. For example, statistical commandsmay be commands known to produce one or more statistical values as anoutput. A statistical value may also be referred to herein as a summarystatistic where that statistical value summarizes or is based on astatistical function that takes into account multiple values of dataitems of an event attribute.

In addition to the above described options, those options describedabove in the section 2.3 entitled “EXEMPLARY OPTIONS” and elsewhere mayalso be utilized in these summary implementations where appropriate.

4.4 Hybrid Table Format with Data Summary View

FIG. 19C depicts a hybrid table format with data summary view 1952,hereinafter merely referred to as hybrid view 1952. In such a hybridview some columns may be depicted in the table format while othercolumns are depicted as summary reports of the respective eventattribute of that column. In embodiments, each column can optionally beselectable as to whether or not the user would like to view the columnin a table format or a data summary view. Such a selection may be madein any suitable manner; however, as depicted here, the columns areselectable through checkboxes 1934 a-1934 f. As depicted, columns 1934a, 1934 b, and 1934 e have been selected to be displayed as summaryreports of a data summary view while columns 1934 c, 1934 d, and 1934 fare displayed in a table format view. Another exemplary mechanism forselection would be through highlighting, as described above in referenceto selection of cells, summary entries, or sections of summary graphs.It will be appreciated that these mechanisms for selection are merelymeant to be illustrative of possible mechanisms and should not betreated as limiting. Any suitable mechanism for selection is explicitlycontemplated herein.

Hybrid view 1952 can optionally be activated via a control, such ascontrol 1910 of FIG. 19A and the full table format may be returned viacontrol 1920. In addition, when the hybrid view is activated, selectionof a column can act to automatically convert the display of that columnfrom a table format to a data summary view, or vice versa. In addition,in some implementations, each column could have a control similar tocontrol 1910, discussed above. In such an implementation, the summaryreport of each individual column, or event attribute, may be displayedby activating such a control. In addition, in such an implementation,the system may be configured to cause each summary report to have acontrol, such as control 1920 discussed above, to enable a user toswitch individual columns displaying a summary report to a table format.

It will be appreciated that in some implementations, the system may beconfigured to enable the above discussed interactions with a datasummary view to be carried out on those columns selected to display asummary report. In addition, the system may be configured to enable theinteractions in the described above in the section 2.2 entitled“EXEMPLARY INTERACTIONS WITH A TABLE FORMAT” and elsewhere to be carriedout on those columns displayed in a table format.

4.5 Filtering Events Through Interaction with a Data Summary View

As mentioned previously, in some implementations, a user can interactwith the summary entries and/or the summary graphs depicted in thesummary reports of a data summary view to enable filtering of the set ofevents that satisfy search query 1914. FIG. 19D depicts an illustrativeimplementation of a data summary view that is configured to support suchfiltering. To accomplish this, the system can be configured to enable auser to make a selection of a portion of a summary graph (e.g., value orrange) or a summary entry of a respective summary report. This selectionmay be accomplished through any conventional manner, including, but notlimited to, mouse-over, left-click, right-click, touch selection, etc.Based on the selection, the system can be configured to cause the set ofevents to be filtered based on the corresponding selection. As can beseen in data summary view 1954, each of the summary entries depicted inthe summary reports have a respective checkbox associated therewith(e.g., checkboxes 1936). It will be appreciated that these checkboxesare merely meant to be illustrative of a possible selection mechanismand that any other suitable mechanism for selection may be utilized inaddition to, or in place of, the depicted checkboxes.

In implementations, the system may be configured to detect a userselection (e.g., selection 1938) of a summary entry via the depictedcheckboxes. As used herein, selection may include initial selection of asummary entry, or subsequent selection of the summary entry, such thatsubsequent selection of the summary entry acts to deselect, or uncheck,the summary entry. In response to the selection of the summary entry,the system may be configured to filter the set of events based on theselection. To filter a set of events is to select events of the set ofevents that satisfy the filter. This is as opposed to modifying searchquery 1914 and executing the modified search query to produce a new setof events. As such, filtering the set of events can be more efficientthan modifying search query 1914 and can enable additional uses, suchas, for example, faceted browsing of the set of events to bettervisualize the data. As an example, data summary view depicts a userhaving selected ‘Account DEF’ of the event attribute AccountName.Application of a filter based on this selection would act to limit theevents of the set of events to those events that have ‘Account DEF’ asthe value of a data item within an event attribute. In some embodiments,filtering of a set of events may include applying a late binding schema,such as that discussed elsewhere herein, to the set of events. In suchan embodiment, the late binding schema may be associated with one ormore extraction rules that are based on the user selections. Inaddition, in some embodiments, the system could calculate an affinity orco-occurrence between two or more selected summary entries and displaythe affinity or co-occurrence to the user to aid the user's choice onwhether or not to filter based on the two or more selected values.

Once the set of events has been filtered based on the user selection, insome embodiments, the system is configured to update the summary graphsto reflect the filtered set of events. In such embodiments, as the usersuccessively selects summary entries, the user can gain additionalinsight into the data through the changes reflected in the summarygraphs in response to each selection. As an example, summary graph 1922c has been updated to reflect, or correspond with, the filtered set ofevents that correspond with selection 1938. As such, summary graph 1922c reflects that only those events that have ‘Account DEF’ as the valueof a data item within an event attribute are included within thefiltered set of events. In addition, summary graph 1922 d has also beenupdated to reflect the impact of the filtered set of events to thedistribution of values over time for the AccountRegion event attribute.As can be seen, the section of this summary graph that depicted the‘West’ value in FIGS. 19B and 19C has been removed from summary graph1922 d of FIG. 19D. This reflects that the filtered set of events thatcorrespond with selection 1938 does not include any events that have avalue of ‘West’ for the AccountRegion event attribute. It will beappreciated, however, that all summary graphs, or any subset thereof,could reflect changes to the distribution of values for the respectivelyassociated event attribute over time. These changes could includeadjustments to the distribution of values over time as well as removalof those sections of the graph that are no longer reflected in thefiltered set of events.

In some embodiments, the summary entries may also be updated to reflectthe filtered set of events. This is demonstrated by the removal of the‘West’ value from summary entries 1940 in response to selection 1938. Inother embodiments, the summary entries that are no longer in thefiltered set of events may remain static or be designated in another way(e.g., change in font color, strikethrough, etc.) such that the user isstill able to select those summary entries for inclusion in the filteredset of events. Such an inclusion could effectively act as an ‘or’ in thefiltered set of events. In further embodiments, however, the summaryentries for the event attribute associated with the selection (e.g.,selection 1938) may remain static, regardless of the manner of updatingthe summary entries for the remaining event attributes. Such anembodiment would allow for further selection of values for theassociated event attribute.

In some embodiments, based on the selection, the system can beconfigured to cause one or more options to be displayed (e.g., list ofoptions 1918 discussed previously) corresponding to the selection. Thefiltering of the set of events could then be further based on the userselecting one of the displayed options. Such options are discussed indetail in reference to the above discussed section “ILLUSTRATIVE OPTIONSWITHIN THE DATA SUMMARY VIEW,” however, rather than adding commands tothe search query, the options would be applied to the set of events.

In some embodiments, it may be desirable for a user to be able to savethe filter. For example, once the user has arrived at a filter thatreflects an interesting find in the data the user may wish to save thefilter for application of the filter to a different search query or tothe same search query at a later time that could reflect newly addedevents. In such embodiments, the system may be configured to enable theuser to save the filter via selection of save filter control 1942. Uponselecting save filter control 1942, the user may be able to enter a namefor the filter so that the filter could be opened and applied to adifferent search query, or applied later to newly added events of theexisting search query 1914. In embodiments where the user is interestedin applying the filter at a later time to any newly added events thatsatisfy search query 1914, the filter could be saved as in metadataassociated with a saved version of search query 1914 (e.g., metadataassociated with a saved pipeline, as discussed above) so that the filtercan be applied to search query 1914 when executed in the future (e.g.,when loading the saved pipeline into a search interface).

In some embodiments, it may be desirable for a user to be able to updatethe commands of search query 1914 to reflect the filter that has beenapplied to the set of events. For example, once the user has arrived ata filter that reflects an interesting find in the data, the user maywish to update the commands of the search query to reflect the filter.Such an embodiment would result in an updated set of events that couldthen be further filtered, for instance. In such embodiments, the systemmay be configured to enable the user to update search query 1914 toreflect the filter via selection of control 1944. Upon selecting control1944, the system may generate commands to be added to search query 1914.The system may then add these commands to search query 1914 (and one ormore corresponding command entries to the command entry list) and mayalso automatically update the search results to reflect the updatedsearch query.

It will be appreciated that, in some embodiments, the above discussedfiltering could be applied to the hybrid view discussed above. In suchembodiments, those columns displayed in the table format could also beupdated to reflect the filtered set of events, in addition to updatingthe summary graphs and/or the summary entries displayed in those columnsthat have been selected to display summary reports for the respectiveevent attribute.

In addition, it will be appreciated that, in some embodiments, thesystem may be configured to enable a user to select summary entriesand/or values or ranges within the summary graphs to apply a filter tothe set of events in accordance with the selections, while alsosimultaneously allowing a user to modify the search query through suchselections, as discussed in the section “ILLUSTRATIVE INTERACTIONS WITHDATA SUMMARY VIEW,” above. In such an embodiment, the manner ofselection, for instance, may indicate whether the user intends to applythe selection as a filter or as a change to search query 1914. As anexample, selection of the text of the summary entry (e.g., by clickingon the text) could indicate that the user wishes to modify search query1914, while selection of, for example, a checkbox could indicate thatthe user wishes the selection to be applied as a filter. In suchembodiments, as the system updates the set of events in response tochanges to search query 1914, the filtered events may subsequently beautomatically updated by the system to reflect the updated set ofevents.

5.0 Additional Exemplary Implementations

FIG. 20 presents a flowchart illustrating utilizing interface templatesfor query commands in accordance with the disclosed embodiments. Eachblock illustrating methods in accordance with FIG. 20, and other methodsdescribed herein, comprises a computing process that may be performedusing any combination of hardware, firmware, and/or software. Forinstance, various functions may be carried out by a processor executinginstructions stored in memory. The methods may also be embodied ascomputer-usable instructions stored on computer storage media. Themethods may be provided by a standalone application, a service or hostedservice (standalone or in combination with another hosted service), or aplug-in to another product, to name a few.

At block 2002, events are displayed in a table. For example, a searchsystem can cause display of events that correspond to search results ofa search query in a table (e.g., 1202). The table can include aplurality of rows, each row representing an event of the events, theevents comprising data items of event attributes. The table can alsoinclude a plurality of columns (e.g., 1204 a, 1204 b) forming cells withthe plurality of rows, each column representing a respective eventattribute of the event attributes, the cells of the column displayingthe data items of the respective event attribute. The table further caninclude interactive regions of the table (e.g., cells, columns, rows,and/or portions thereof), each interactive region corresponding to oneor more data items of the displayed data items and being selectable by auser to cause display of a list of options (e.g., 1220 a, 1220 b)corresponding to the selected interactive region.

At block 2004, in response to a user selecting an interactive region ofthe table, a list of options is displayed. For example, in response tothe user selecting the designated interactive region (e.g.,corresponding to 1204 a) of the interactive regions of the table, thesearch system can cause display of the list of options (e.g., 1220 a,1220 c) corresponding to the designated interactive region, eachdisplayed option corresponding to an interface template for composingquery commands.

At block 2006, based on the user selecting an option, commands are addedto the query as composed according to an interface template. Forexample, based on the user selecting an option (e.g., 1220 a) in thedisplayed list of options, causing one or more commands (e.g.,corresponding to command entry 1232 b) to be added to the search query,the one or more commands composed based on the one or more data itemsthat corresponds to the designated interactive region according toinstructions of the interface template of the selected option.

In some cases, the one or more commands are composed based on input fromthe user into the one or more form elements of a form, the instructionsof the interface template mapping the one or more form elements to oneor more portions of the one or more commands.

In some cases, the method further comprises based on the user selectingan option in the displayed list of options, causing display of agraphical user interface defined by the instructions of the interfacetemplate, and composing the one or more commands from user input to thegraphical user interface.

In some cases, the method further comprises receiving a request from theuser to modify the one or more commands added to the search query, inresponse to the request, causing a form to be presented comprising oneor more form elements defined by the instructions of the interfacetemplate, and modifying the one or more commands in the search querybased on input from the user to the one or more form elements, theinstructions of the interface template mapping input to one or moreportions of the one or more commands.

In some cases, the instructions of the interface template compriseextracting values of the one or more data items that corresponds to thedesignated interactive region, determining a number of form elements todisplay to the user based on the extracted values, causing display ofeach of the form elements to the user based on the user selecting anoption in the displayed list of options, and composing the one or morecommands based on input from the user into one or more of the formelements.

In some cases, the one or more commands are composed and added to thesearch query based on input from the user into the one or more formelements defined by the instructions of the interface template, and themethod further comprises adding one or more subsequent commands to thesearch query, receiving a selection, by the user of a command entryrepresenting the one or more commands of the search query, and based onthe selection, causing presentation of at least one of the one or moreform elements to the user for modifying the one or more commands addedto the search query, the at least one of the one or more form elementscomprising the input from the user.

In some cases, the one or more commands are composed and added to thesearch query using a form defined by the instructions of the interfacetemplate, and the method further comprises adding one or more subsequentcommands to the search query, receiving a selection, by the user of acommand entry representing the one or more commands of the search query,and based on the selection, causing presentation of the form to the userfor modifying the one or more commands added to the search query.

In some cases, the causing one or more commands to be added to thesearch query automatically causes the displayed events in the table tobe updated to correspond to the search query comprising the one or morecommands.

In some cases, the method further comprises receiving a selection, bythe user of a command entry representing one or more previously addedcommands of the search query, and based on the selection, causing adisplayed form defined by the instructions of the interface template forcomposing the one or more commands to be replaced with a form defined byinstructions of another interface template for composing the one or morepreviously added commands.

In some cases, the method further comprises receiving a selection, bythe user of a command entry representing one or more previously addedcommands of the search query, and based on the selection, causing one ormore form elements defined by the instructions of the interface templatefor composing the one or more commands to be replaced in an interfacepanel with one or more form elements defined by instructions of anotherinterface template for composing the one or more previously addedcommands.

In some cases, the method further comprises causing display of aninterface panel to the user, the interface panel being configured todisplay form elements for composing commands of the search query, inresponse to a request from the user to hide the interface panel, hidingthe displayed interface panel including the form elements, and based onthe user selecting the option in the displayed list of options, causingthe hidden interface panel to be automatically unhidden, and causing theunhidden interface panel to include form elements defined by theinstructions for composing the one or more commands.

In some cases, the instructions define that each command composed usingthe interface template identifies the one or more data items thatcorresponds to the designated interactive region in the command.

In some cases, the instructions define that each command composed usingthe interface template is composed by determining an identifier of anevent attribute of the one or more data items that corresponds to thedesignated interactive region, and including the identifier in thecommand.

In some cases, the instructions specify a command identifier to includein each command composed using the interface template.

In some cases, the search query is executed on event data that includesa plurality of events, each event including a timestamp associated withraw data.

In some cases, the method further comprises executing the search query,wherein the executing returns the search results comprising a pluralityof events, the executing performing an extraction of values from theplurality of events using a common extraction rule, the extractiondefined by the second query.

In some cases, the method further comprises executing the search query,wherein the executing returns the search results comprising a pluralityof events, the executing performing an extraction of values from theplurality of events using a regular expression, the extraction definedby the second query.

In some cases, the method further comprises executing the search query,wherein the executing returns query results comprising a plurality ofevents, the executing applying a late-binding schema to the plurality ofevents.

In some cases, the search query is represented in a pipeline querylanguage and the one or more commands are written in the pipelined querylanguage.

FIG. 21 presents a flowchart illustrating adding supplemental eventattributes to a table format in accordance with the disclosedembodiments.

At block 2102, events are displayed in a table. For example, a searchsystem can cause display of events that correspond to search results ofa search query, the events comprising data items of event attributes,where the events are displayed in a table (e.g., 1202). The table caninclude cells arranged in rows and columns, each column (e.g., 1233 c)corresponding to an event attribute the event attributes, and each rowcorresponding to an event of the events, the cells of each column beingpopulated with the data items that correspond to the event attribute ofthe column, and interactive regions of the table (e.g., columns, cells,rows, and/or portions thereof), each interactive region corresponding toat least one data item of the displayed data items and being selectableby a user to add one or more commands to the search query based on theselected interactive region.

At block 2104, a reference event attribute is determined based on ananalysis of a data object. For example, the search system can determinea reference event attribute (e.g., 1230) of the event attributes in thedisplayed table for adding a supplemental event attribute (e.g.,corresponding to 1233 a, 1233 b) to the event attributes displayed inthe table based on an analysis of a data object (e.g., corresponding to1102 c) that defines the supplemental event attribute.

At block 2106, a supplemental column is added corresponding to asupplemental event attribute of the table. For example, the searchsystem can based on the user selecting an option (e.g., 1232), performthe adding comprising causing a supplemental column (e.g., 1233 a, 1233b) corresponding to the supplemental event attribute to be added to thedisplayed table, cells of the supplemental column being populated withsupplemental data items, each supplemental data item being mapped to arespective event for the supplemental column using a data item of thereference event attribute of the respective even and being displayed inthe row corresponding to the respective event.

At block 2108, interactive regions are added to the table correspondingto data items of the supplemental event attribute. For example, theadding can further comprise the search system causing supplementalinteractive regions (e.g., columns, cells, and/or portions thereof) tobe added to the interactive regions of the displayed table, eachsupplemental interactive region corresponding to at least one of thesupplemental data items and being selectable by the user to add one ormore commands to the search query based on the selected supplementalinteractive region.

In some cases, the performing the adding further comprises causing thesearch query to be updated to include one or more commands, the updatedsearch query being executable to return search results comprising eventshaving data items of the event attributes and the supplemental eventattribute.

In some cases, the reference event attribute if of a plurality ofreference event attributes identified based on the analysis of the dataobject, and the method further comprises causing presentation of theplurality of identified reference event attributes to the user asoptions for the adding of the supplemental column, where the optionselected by the user is included in the options.

In some cases, each supplemental data item that is mapped to therespective event for the supplemental column is determined from the dataobject using a common combination type assigned to the adding.

In some cases, each supplemental data item that is mapped to therespective event for the supplemental column is determined from the dataobject using a join combination type identified for the adding.

In some cases, each supplemental data item that is mapped to therespective event for the supplemental column is determined from the dataobject using a lookup combination type identified for the adding.

In some cases, the method further comprises identifying a combinationtype preconfigured for the data object by another user, and based on theidentifying of the combination type, using the combination type for theadding, wherein each supplemental data item that is mapped to therespective event for the supplemental column is determined from the dataobject using the combination type identified for the adding.

In some cases, t each supplemental data item is extracted fromrelational data identified by the data object.

In some cases, each supplemental data item is extracted from a lookuptable identified by the data object.

In some cases, each supplemental data item is extracted from searchresults of a supplemental search query identified by the data object.

In some cases, the adding further comprises executing a supplementalsearch query associated with the data object and each supplemental dataitem is extracted from search results of the supplemental search query.

In some cases, each supplemental data item of the supplemental eventattribute is extracted from at least one data source external to thesearch query.

In some cases, the data object defines a plurality of supplemental eventattributes including the supplemental event attribute and an additionalsupplemental event attribute, and the performing the adding furthercomprises causing an additional supplemental column corresponding to theadditional supplemental event attribute to be added to the displayedtable, cells of the additional supplemental column being populated withadditional supplemental data items, each additional supplemental dataitem being mapped to a respective event for the additional supplementalcolumn using a data item of the reference event attribute of therespective even and being displayed in the row corresponding to therespective event, and causing additional supplemental interactiveregions to be added to the interactive regions of the displayed table,each additional supplemental interactive region corresponding to atleast one of the additional supplemental data items and being selectableby the user to add one or more commands to the search query based on theselected additional supplemental interactive region.

In some cases, the data object defines a plurality of supplemental eventattributes, and the method further comprises causing presentation of theplurality of supplemental event attributes to the user based on thedetermining of the reference event attribute, receiving, from the user,a selection of a subset of the presented plurality of supplemental eventattributes for the adding, the subset including the supplemental eventattribute, and based on the selection of the subset, performing theadding wherein a supplemental column and corresponding supplemental dataitems is added to the displayed table for each supplemental eventattribute in the subset.

In some cases, the determining the reference event attribute of theevent attributes in the displayed table comprises identifying apotential event attribute defined by the data object, the potentialevent attribute having a plurality of values, calculating a level ofsimilarity between one or more values of the plurality of values of thepotential event attribute and one or more of the data items of aparticular event attribute of the event attributes displayed in thetable based on an analysis of the one or more values, and selecting theparticular event attribute as the reference event attribute based on thecalculated level of similarity, where the potential event attribute isused as the supplemental event attribute.

In some cases, the determining the reference event attribute of theevent attributes in the displayed table comprises, identifying apotential event attribute defined by the data object, the potentialevent attribute having an attribute label, comparing the attribute labelof the potential event attribute to an attribute label of a particularevent attribute of the event attributes displayed in the table, andselecting the particular event attribute as the reference eventattribute based on the comparing, where the potential event attribute isused as the supplemental event attribute.

In some cases, the search query is executed on event data that includesa plurality of events, each event including a timestamp associated withraw data.

In some cases, the search query defines an extraction of values from aplurality of events using a common extraction rule, the reference eventattributes comprising the extracted values.

In some cases, the search query defines an extraction of values from aplurality of events using a regular expression, the reference eventattributes comprising the extracted values.

In some cases, the search query applies a late-binding schema to theevents.

In some cases, the search query is represented in a pipeline querylanguage and the performing the adding further comprises adding one ormore pipelined commands to the search query, the updated search querybeing executable to return search results comprising events having dataitems of the event attributes and the supplemental event attribute.

FIG. 22 presents a flowchart illustrating utilizing runtime permissionsfor queries in accordance with the disclosed embodiments. At block 2202,an access permission of a first user is assigned to a first query as aruntime permission. For example, a search system can assign an accesspermission of a first user to a query object that represents a firstquery (e.g., 1604), the access permission granting the first user accessrights to one or more data sources (e.g., 1654) of the first query, theaccess permission being assigned as a runtime permission of the firstquery.

At block 2204, a request is granted from a second user to execute asecond query. For example, the search system can grant a request from asecond user to execute a second query (e.g., 1602).

At block 2206, an access request is received to execute the first queryas a subquery of the second query. For example, the search system canreceive an access request directed to the query object to execute thefirst query as a subquery of the second query.

At block 2208, the access request is granted to allow the second user toexecute the first query as the subquery using the runtime permission.For example, based on the granting of the request from the second userto execute the second query, the search system can grant the accessrequest, the granting of the access request allowing the second user toexecute the first query on the one or more data sources (e.g., 1654) ofthe first query using the runtime permission assigned to the first queryin executing the second query using the first query as the subquery.

In some cases, the method further comprises assigning an accesspermission of a third user to a base query object, the access permissiongranting the third user access rights to one or more data sources of thethird query, the access permission being assigned as a runtimepermission of the third query, and allowing the second user to executethe second query on the one or more data sources of the third queryusing the runtime permission assigned to the third query.

assigning an access permission of a third user to a base query object,the access permission granting the third user access rights to one ormore data sources of the second query, the access permission beingassigned as a runtime permission of the second query, and based on thegranting of the request from the second user to execute the secondquery, allowing the second user to execute the second query on the oneor more data sources of the second query using the runtime permissionassigned to the second query.

In some cases, the allowing the second user to execute the first queryon the one or more data sources of the first query for the subqueryusing the runtime permission assigned to the first query returnssupplemental query results of the first query to a data processingpipeline corresponding to the second query.

In some cases, the first query is an input query of the second query,the input query representing an initial portion of a data processingpipeline represented by the second query.

In some cases, the one or more data sources of the first query compriseone or more indexed data stores and the first query is on indexedobjects in the one or more data indexed data stores.

In some cases, the request from the second user to execute the secondquery is directed to another query object that represents the secondquery to execute the second query as a subquery of a third query.

In some cases, the granting the request from the second user to executethe second query is based on the second user having access rights toanother query object that represents the second query.

In some cases, the second query defines a plurality of subqueries, eachsubquery having a respective query object that defines runtimepermissions of the subquery that are used for executing the subquery inthe executing of the second query.

In some cases, the method further comprises constructing, by the seconduser, the second query in a query interface, wherein the request fromthe second user to execute the second query is received in the queryinterface and causes query results of the second query to be displayedin the query interface.

In some cases, the method further comprises constructing, by the firstuser, the first query using a query interface configured to displayquery results of the first query to the first user, receiving, by thequery interface, a request from the first user to save the first query,and saving the first query in association with the query object based onthe request from the first user, the saving comprising the assigning ofthe access permission of the first user to the query object.

In some cases, the method further comprises constructing, by a thirduser, a third query using a query interface configured to display queryresults of the third query to the third user, receiving, by the queryinterface, a request from the third user to save the third query, andsaving the third query in association with another query object based onthe request from the third user, the saving comprising assigning anaccess permission of the third user to the another query object as aruntime permission of the third query. The granting the request from thesecond user to execute the second query allows the second user toexecute the third query on the one or more data sources of the thirdquery as a base query to the second query using the runtime permissionassigned to the third query.

In some cases, the runtime permission is designated by the query objectas being for a project, and the granting the access request is based onthe executing of the second query being for the project.

In some cases, the first query and the second query are executed onevent data that includes a plurality of events, each event including atimestamp associated with event raw data.

In some cases, the executing of the second query returns query resultscomprising a plurality of events, the executing performing an extractionof values from the plurality of events using a common extraction rule,the extraction defined by the second query.

In some cases, the executing of the second query returns query resultscomprising a plurality of events, the executing performing an extractionof values from the plurality of events using a regular expression, theextraction defined by the second query.

In some cases, the executing of the second query returns query resultscomprising a plurality of events, the executing applying a late-bindingschema to the plurality of events.

In some cases, the first query and the second query each comprise aplurality of pipelined commands of a pipelined query language.

FIG. 23 presents a flowchart illustrating integrating query interfacesin accordance with the disclosed embodiments. At block 2302, a query isreceived in a first query interface. For example, a search system canreceive, in a first query interface (e.g., corresponding to FIG. 14B), aquery composed by the user by typing commands into a query box (e.g.,1402) of the first query interface.

At block 2304, events are displayed corresponding to the query withfields corresponding to the events. For example, based on the receivingof the query, the search system can cause events corresponding to queryresults of the query to be displayed (e.g., in 1408) in the first queryinterface with fields (e.g., in 1406 and/or 1408) corresponding to theevents.

At block 2306, a selection is received of an option to load the queryinto a second query interface. For example, the search system canreceive, from the user, a selection of an option (e.g., 1442) to loadthe query composed in the first query interface into a second queryinterface (e.g., corresponding to FIG. 15).

At block 2308, the second query interface is displayed with a tablecomprising events corresponding to the loaded query. For example, basedon the selection by the user of the option, the search system can causedisplay of the second query interface and display of a table (e.g.,1502) in the second query interface that comprises events thatcorrespond to query results of the loaded query (e.g., represented in1508), the events comprising data items of event attributes. Thedisplayed table can comprise cells arranged in rows and columns, eachcolumn corresponding to an event attribute the event attributes, andeach row corresponding to an event of the events, the cells of eachcolumn being populated with the data items that correspond to the eventattribute of the column, wherein the event attribute that corresponds toa designated column of the columns is a field of the fields displayed inthe first query interface, and interactive regions of the table (e.g.,columns, cells, rows, and/or portions thereof), each interactive regioncorresponding to at least one data item of the displayed data items andbeing selectable by the user to add one or more commands to the loadedquery based on the selected interactive region.

In some cases, the method further comprises based on the selection bythe user of the option, causing display of one or more form elements,each form element corresponding to at least one of the fields, the formelements allowing the user to select a subset of the fields, where thedisplay of the table includes a respective column in the columns of thetable for each selected field in the subset of the fields based on theuser selecting the subset of the fields, and the event attribute of therespective column is the selected field.

In some cases, the method further comprises based on the selection bythe user of the option, causing display of one or more form elements,each form element corresponding to at least one of the fields, the formelements allowing the user to select a subset of the fields, and causingat least one column to be added to the table such that the tableincludes a respective column in the columns of the displayed table foreach selected field in the subset of the fields based on the userselecting the subset of the fields, the event attribute of therespective column being the selected field.

In some cases, the method further comprises based on the selection bythe user of the option, causing one or more commands to be automaticallyadded to the query to result in the loaded query, the one or morecommands causing at least one of the fields to be removed from theevents that correspond to query results of the loaded query.

In some cases, the query box is a search bar and the query is a searchquery.

In some cases, the field is defined in the first query interface outsideof the query by an extraction rule.

In some cases, the method further comprises receiving, in the firstquery interface, a selection by the user of the field from the fieldsdisplayed with the events in the first query interface, and in responseto the selection of the field, causing values of the field to bedisplayed in association with the events, each value being displayedwith a corresponding one of the events. The designated column isincluded in the table of the second query interface based on theselection of the field in the first query interface.

In some cases, the method further comprises receiving, in the firstquery interface, a selection by the user of a designated field from thefields displayed with the events in the first query interface, inresponse to the selection of the designated field, and causing values ofthe designated field to be remove from display in association with theevents, each value displayed with a corresponding one of the events. Thedisplay of the table in the second query interface is caused toautomatically exclude a designated column for the designated field basedon the selection by the user of the designated field.

In some cases, the method further comprises based on the selection bythe user of the option, causing display of a command entry list, atleast one command entry in the command entry list representing the querycomposed in the first query interface.

In some cases, the option selected by the user is a button in the firstquery interface and the selection of the button causes the first queryinterface to transition to the second query interface.

In some cases, the method further comprises causing an update to thequery to include one or more commands based on the selection by the userof one or more of the interactive regions of the table in the secondquery interface, causing the updated query to be saved in associationwith a query object, receiving a selection of a pipeline entryrepresenting the query object in a selection interface, and in responseto the selection of the pipeline entry, loading the updated query intothe first query interface using the query object.

In some cases, the method further comprises causing an update to thequery to include one or more commands based on the selection by the userof one or more of the interactive regions of the table in the secondquery interface, causing the updated query to be saved in associationwith a query object, and loading the updated query into the first queryinterface using the query object, where at least a portion of theupdated query is represented in the query box.

In some cases, the method further comprises causing an update to thequery to include one or more commands based on the selection by the userof one or more of the interactive regions of the table in the secondquery interface, causing the updated query to be saved in associationwith a query object, the updated query comprising saved commands, andloading the updated query into the first query interface using the queryobject, the updated query being loaded as an input query for adesignated user, where the first query interface allows the user to addone or more commands to the saved commands of the updated query saved inassociation with the query object, and precludes the designated userfrom modifying the saved commands of the updated query saved inassociation with the query object.

In some cases, the method further comprises based on the selection bythe user of the option, automatically modifying the query such that theloaded query removes one or more of the fields displayed in the firstquery interface from the query results of the loaded query.

In some cases, the query is executed on event data that includes aplurality of events, each event including a timestamp associated withraw data.

In some cases, the method further comprises the query defines anextraction of values from a plurality of events using a commonextraction rule, the values corresponding to the data items in the cellsfor at least one of the columns in the table.

In some cases, the method further comprises the query defines anextraction of values from a plurality of events using a regularexpression, the values corresponding to the data items in the cells forat least one of the columns in the table.

In some cases, the loaded query applies a late-binding schema to theevents that correspond to the query results of the loaded query.

In some cases, the loaded query is represented in a pipelined querylanguage and each interactive region is selectable by the user to addone or more pipelined commands to the loaded query based on the selectedinteractive region.

FIG. 24 presents a flowchart illustrating utilizing a data summary viewin accordance with the disclosed embodiments. In embodiments, theprocess flow may begin at block 2402 where the system receives a requestto display a data summary view (e.g., the data summary view discussedextensively in reference to FIGS. 19A-19D) of search results of a searchquery (e.g., search query 1914 of FIG. 19A). In embodiments, thisrequest may be received while the search results are being displayed ina table format (e.g., table format 1902 of FIG. 19A) within a searchscreen (e.g., search screen 1900 of FIG. 19A). Such a table format mayinclude a plurality of rows and a plurality of columns. Each row of theplurality of rows may represent an event, of a set of events thatsatisfy the search query. Thea plurality of columns may form cells withthe plurality of rows. Each column may represent a respective eventattribute of a plurality of event attributes of the set of events andmay include data items of the respective event attribute populating onesof the cells (e.g., cell 1912 of FIG. 19A).

At block 2404, in response to receiving the request, the system maycause display of the requested data summary view. The data summary viewmay include a summary report, such as that discussed extensively inreference to FIG. 19A-19D, for a selected event attribute of theplurality of event attributes. The summary report may include summaryentries (e.g., 1924 b, 1924 f, 1926, and 1928 of FIG. 19B) that presenta summary of data items of the selected event attribute. In addition,the summary report may include a summary graph (e.g., 1922 a and 1922 bof FIG. 19B or 1922 c and 1922 d of FIG. 19d ) of these data items. Thesummary graph may depict a distribution of at least a subset of valuesof the data items of the selected event attribute over a period of time.In some cases a format of the summary entries and the summary graph arebased on a type (e.g., numerical, categorical, user defined, etc.)associated with the selected event attribute. In some cases, the summaryentries may identify one or more values within the data items of theselected event attribute that occur above an upper occurrence threshold(e.g., top values 1924 f of FIG. 19B). In some cases, the summaryentries may identify one or more values within the data items of theselected event attribute that occur below a lower occurrence threshold(e.g., rare values 1928 of FIG. 19B). In some cases, the summary entriesidentify one or more values within the data items of the selected eventattribute that are beyond a threshold of similarity from other valueswithin the data items of the selected event attribute (e.g., unusualvalues 1926 of FIG. 19B). In some cases, the summary entries includesummary statistics, such as those summary statistics discussedpreviously (e.g., summary statistics represented by summary entries 1924b of FIG. 19B). In some cases the summary entries are based on a datatype (e.g., numerical, categorical, user defined, etc.) of the selectedevent attribute. In such cases, for a numeric data type the summaryentries may include one or more of a maximum value, a minimum value, amean value, a median value, a mode value, and a standard deviation. Insome cases, the summary entries for a numeric data type, for example,could include one or more of: a maximum value, a minimum value, a meanvalue, a median value, a mode value, and a standard deviation.

In some cases, the summary graph depicts the distribution of values ofthe subset of the data items of the selected event attribute over aperiod of time. In such cases, the distribution of values may includeone or more of a maximum value, a minimum value, a standard deviation,and an average value.

In some cases, the data summary view may include a separate summaryreport for each of one or more additional event attributes of theplurality of event attributes, up to and including all of the pluralityof event attributes (e.g., data summary view 1946). As with the abovediscussed summary report, each of the additional summary reports mayinclude summary entries and/or a summary graph of data items of arespective one of the one or more additional event attributes.

In some cases, in causing display of the data summary view the systemmay be configured to cause the data summary view to overlay the tableformat. In some cases this overlay could be displayed over one or morecolumns of the table. In such cases, a portion of the table format maystill be viewed alongside the overlayed data summary view.

In some cases, the system may enable a user to switch from the datasummary view to the table format from within the data summary view andswitch from the table format to the data summary view from within thetable format.

In some cases, causing display of the data summary view includes causingthe data summary view to overlay one or more columns of the tableformat. In such cases, the one or more columns would include the columnthat represents the selected event attribute. In addition, such casesmay include overlaying the one or more columns of the table format suchthat a remainder of the table format is still visible. In some cases,each of the plurality of columns is selectable by a user (e.g., viacheckboxes 1934 a-1934 f of FIG. 19C). In such cases the request todisplay a data summary view may include an identifier of one or morecolumns that have been selected by the user (e.g., columns 1904 a, 1904b, and 1904 e of FIG. 19C). In such cases the one or more columns thatoverlay the table format can be the columns identified in the request.

In some cases the data summary view is interactive such that the datasummary view includes user selectable portions (e.g., selectable summaryentries and/or selectable values or ranges of values from the summarygraph). In such cases, the system may be configured to receive aselection of a selectable portion of the data summary view. In responseto receiving the selection of the selectable portion the system mayupdate the search query based on the selected portion.

In some cases, the system may be configured to, upon receiving theselection of the selectable portion of the data summary view, cause alist of options (e.g., options menu 1918 of FIG. 19B) to display to theuser. In some cases, the list of options may be dependent on a type(e.g., time, numerical, categorical, user defined, etc.) of theselectable portion selected. Such a list of options is discussedextensively above. The system may then, update the search query (e.g.,by causing one or more commands to be added to the search query). Insome cases, the one or more commands may be based, at least in part, onan option that is selected from the list of options. In some cases, thesystem may be configured to automatically update the set of events thatsatisfy the search query to correspond with the updated search query.

FIG. 25 presents a flowchart illustrating utilizing a data summary viewwith filtering in accordance with the disclosed embodiments. Suchfiltering can enable faceted browsing of the set of events. In someembodiments, the process flow may begin where the system receives arequest to display a data summary view (e.g., the data summary viewdiscussed extensively in reference to FIGS. 19A-19D) of search resultsof a search query (e.g., search query 1914 of FIG. 19A), as discussed inreference to FIG. 24, above. In some embodiments, this request may bereceived while the search results are being displayed in a table format(e.g., table format 1902 of FIG. 19A) within a search screen (e.g.,search screen 1900 of FIG. 19A). Such a table format may include aplurality of rows and a plurality of columns. Each row of the pluralityof rows may represent an event, of a set of events that satisfy thesearch query. The plurality of columns may form cells with the pluralityof rows. Each column may represent a respective event attribute of aplurality of event attributes of the set of events and may include dataitems of the respective event attribute populating ones of the cells(e.g., cell 1912 of FIG. 19A).

In other embodiments, the process may begin at block 2502, where thesystem may cause display of the data summary view. The data summary viewmay include summary reports, such as the summary reports discussedextensively in reference to FIGS. 19A-19D, that are each respectivelyassociated with a selected event attribute. The summary reports may eachinclude summary entries (e.g., 1924 b, 1924 f, 1926, and 1928 of FIG.19B) that present a summary of data items of the respectively associatedevent attribute. In addition, the summary reports may each include asummary graph (e.g., 1922 a and 1922 b of FIG. 19B or 1922 c and 1922 dof FIG. 19d ) of the respective data items of the respectively selectedevent attribute. The summary graphs may depict a distribution of atleast a subset of values of the respective data items over a period oftime.

In some cases a format of the summary entries and the summary graphs ofthe summary reports are based on a type (e.g., numerical, categorical,user defined, etc.) associated with the respective event attribute. Insome cases, the summary entries may identify one or more values withinthe data items of the respective event attribute that occur above anupper occurrence threshold (e.g., top values 1924 f of FIG. 19B). Insome cases, the summary entries may identify one or more values withinthe data items of the respective event attribute that occur below alower occurrence threshold (e.g., rare values 1928 of FIG. 19B). In somecases, the summary entries identify one or more values within the dataitems of the respective event attribute that are beyond a threshold ofsimilarity from other values within the data items of the respectiveevent attribute (e.g., unusual values 1926 of FIG. 19B). In some cases,the summary entries include summary statistics, such as those summarystatistics discussed previously (e.g., summary statistics represented bysummary entries 1924 b of FIG. 19B). In some cases the summary entriesare based on a data type (e.g., numerical, categorical, user defined,etc.) of the respective event attribute. In such cases, for a numericdata type the summary entries may include one or more of a maximumvalue, a minimum value, a mean value, a median value, a mode value, anda standard deviation.

In some cases, the summary graphs of the summary reports depict thedistribution of values of the subset of the data items of the respectiveevent attributes over a period of time. In such cases, the distributionof values may include one or more of a maximum value, a minimum value, astandard deviation, and an average value.

In some cases, in causing display of the data summary view the systemmay be configured to cause the data summary view to overlay the tableformat. In some cases this overlay could be displayed over one or morecolumns of the table. In such cases, a portion of the table format maystill be viewed alongside the overlayed data summary view.

In some cases, the system may enable a user to switch from the datasummary view to the table format from within the data summary view andswitch from the table format to the data summary view from within thetable format.

In some cases, causing display of the data summary view includes causingthe data summary view to overlay one or more columns of the tableformat. In such cases, the one or more columns would include the columnsthat represent the respective event attributes of the summary reports.In addition, such cases may include overlaying the one or more columnsof the table format such that a remainder of the table format is stillvisible. In some cases, each of the plurality of columns is selectableby a user (e.g., via checkboxes 1934 a-1934 f of FIG. 19C). In suchcases the request to display a data summary view may include anidentifier of columns that have been selected by the user (e.g., columns1904 a, 1904 b, and 1904 e of FIG. 19C). In such cases the columns thatoverlay the table format can be the columns identified in the request.

In some cases the data summary view is interactive such that the datasummary view includes user selectable portions (e.g., checkboxes 1936 ofFIG. 19D and/or selectable values or ranges of values from the summarygraphs). In such cases, the system may be configured to receive aselection of a selectable portion of a first of the summary reports. Inresponse to receiving the selection of the selectable portion the systemmay filter the set of events based on the selected portion at block2504.

In some cases, the system may be configured to, upon receiving theselection of the selectable portion of the first of the summary reports,cause a list of filtering options to display to the user. In some cases,the list of filtering options may be dependent on a type (e.g.,numerical, categorical, user defined, etc.) of the selectable portionselected. Such a list of options is discussed above in the section“FILTERING EVENTS THROUGH INTERACTION WITH THE DATA SUMMARY VIEW.” Oncethe set of events has been filtered, in some cases the system may beconfigured to cause an update the summary graphs, at block 2506, of thesummary reports to reflect the filtered set of events. In some cases,the system may be further configured to cause an update of at least thesummary entries of those summary reports that did not have a selectableportion selected to reflect the filtered set of events. In some casesthe system may be configured to cause the summary entries of the firstsummary report to remain static to allow for additional selection fromthe first summary report which would enable additional filtering. Insome cases, the system may be configured to cause the set if eventsdisplayed in the table view to reflect the filtered set of events.

In some cases, the user may select another selectable portion from adifferent summary report than the summary report that was previouslyselected from. In such cases, the system may be further configured to,in response to selection of one or more of the selectable portions bythe user, update the filter based on the selection. In such cases, thesystem may be configured to refilter the set of events utilizing theupdated filter. In addition, in such cases the system may be configuredto cause an update of at least the summary graphs of the summary reportsto reflect the refiltered set of events.

In some cases, it may be desirable for a user to be able to save thefilter. For example, once the user has arrived at a filter that reflectsan interesting find in the data the user may wish to save the filter forapplication of the filter to a different search query or to the samesearch query at a later time that could reflect newly added events. Insuch embodiments, the system may be configured to enable the user tosave the filter. In some cases, the system may be configured to enablethe user save the filter as metadata associated with a saved version ofthe search query so that the filter can be applied to the search querywhen executed in the future.

In some embodiments, it may be desirable for a user to be able to updatethe commands of the search query to reflect the filter that has beenapplied to the set of events. For example, once the user has arrived ata filter that reflects an interesting find in the data, the user maywish to update the commands of the search query to reflect the filter.Such an embodiment would result in an updated set of events that couldthen be further filtered, for instance. In such embodiments, the systemmay be configured to enable the user to update the search query toreflect the filter via selection of a control (e.g., control 1944 ofFIG. 19D). Upon selecting the control, the system may be configured togenerate commands to be added to the search query. The system may thenadd these commands to the search query and may also automatically updatethe search results to reflect the updated search query.

From the foregoing, it will be seen that this invention is one welladapted to attain all the ends and objects set forth above, togetherwith other advantages which are obvious and inherent to the system andmethod. It will be understood that certain features and subcombinationsare of utility and may be employed without reference to other featuresand subcombinations. This is contemplated by and is within the scope ofthe claims

The invention claimed is:
 1. A computer-implemented method comprising:causing display of a set of events in a table, an event corresponding toa portion of raw machine data associated with a timestamp, the tablecomprising data items of event attributes associated with the set ofevents and an interactive region corresponding to one or more of thedata items; in response to a selection corresponding to the interactiveregion, causing display of one or more form elements that correspond toparameters of one or more commands, at least a form element of the oneor more form elements being based on an event attribute associated withthe interactive region; and causing the one or more commands to be addedto a query, the parameters of the one or more commands composed at leastfrom one or more values of the form element.
 2. The computer-implementedmethod of claim 1, wherein the one or more commands are composed basedon user input into the form element, the form element mapping to one ormore portions of the one or more commands.
 3. The computer-implementedmethod of claim 1, further comprising based on the selection, causingdisplay of a graphical user interface comprising the one or more formelements, wherein the one or more commands are composed from user inputto the graphical user interface.
 4. The computer-implemented method ofclaim 1, further comprising: after adding the one or more commands tothe query, receiving a request to modify the one or more commands addedto the query; in response to the request, causing the one or more formelements to be presented; and modifying the one or more commands in thequery based on user input that modifies the one or more values of theform element.
 5. The computer-implemented method of claim 1, comprising,based on the selection of the interactive region: extracting at leastone value of the one or more data items; and determining a number of theone or more form elements to display based on the at least one value. 6.The computer-implemented method of claim 1, wherein the causing one ormore commands to be added to the query automatically causes the set ofevents displayed in the table to be updated to correspond to results ofthe query comprising the one or more commands.
 7. Thecomputer-implemented method of claim 1, further comprising: receivinguser input corresponding to a command entry representing one or morepreviously added commands of the query; and based on the user input,causing a currently displayed form that includes the one or more formelements to be replaced with a form for modifying the one or morepreviously added commands.
 8. The computer-implemented method of claim1, wherein a set of form elements selectable to compose a set ofcommands of the query are displayed in an interface panel, and themethod further comprises: in response to a user request to hide theinterface panel, hiding the interface panel including the set of formelements; and based on the selection, causing the interface panel to beautomatically unhidden in the display of the one or more form elements.9. The computer-implemented method of claim 1, wherein the display ofthe one or more form elements includes the one or more values in theform element based on the one or more of the data items.
 10. Thecomputer-implemented method of claim 1, wherein the query is representedin a pipelined query language and the one or more commands are writtenin the pipelined query language in which a command of the query producesan output from an input of the command, and the input is an output of aprior command of the query.
 11. One or more non-transitorycomputer-readable media having instructions stored thereon, theinstructions, when executed by a processor of a computing device, tocause the computing device to perform a method comprising: causingdisplay of a set of events in a table, an event corresponding to aportion of raw machine data associated with a timestamp, the tablecomprising data items of event attributes associated with the set ofevents and an interactive region corresponding to one or more of thedata items; in response to a selection corresponding to the interactiveregion, causing display of one or more form elements that correspond toparameters of one or more commands, at least a form element of the oneor more form elements being based on an event attribute associated withthe interactive region; and causing the one or more commands to be addedto a query, the parameters of the one or more commands composed at leastfrom one or more values of the form element.
 12. The one or morecomputer-readable media of claim 11, wherein the one or more commandsare composed based on user input into the form element, the form elementmapping to one or more portions of the one or more commands.
 13. The oneor more computer-readable media of claim 11, wherein the method furthercomprises based on the selection, causing display of a graphical userinterface comprising the one or more form elements, wherein the one ormore commands are composed from user input to the graphical userinterface.
 14. The one or more computer-readable media of claim 11,wherein the method further comprises: after adding the one or morecommands to the query, receiving a request to modify the one or morecommands added to the query; in response to the request, causing the oneor more form elements to be presented; and modifying the one or morecommands in the query based on user input that modifies the one or morevalues of the form element.
 15. The one or more computer-readable mediaof claim 11, wherein the method further comprises, based on theselection of the interactive region: extracting at least one value ofthe one or more data items; and determining a number of the one or moreform elements to display based on the at least one value.
 16. Acomputer-implemented system comprising: one or more processors; andmemory having instructions stored thereon, the instructions, whenexecuted by the one or more processors, to cause the one or moreprocessors to perform a method comprising: causing display of a set ofevents in a table, an event corresponding to a portion of raw machinedata associated with a timestamp, the table comprising data items ofevent attributes associated with the set of events and an interactiveregion corresponding to one or more of the data items; in response to aselection corresponding to the interactive region, causing display ofone or more form elements that correspond to parameters of one or morecommands, at least a form element of the one or more form elements beingbased on an event attribute associated with the interactive region; andcausing the one or more commands to be added to a query, the parametersof the one or more commands composed at least from one or more values ofthe form element.
 17. The system of claim 16, wherein the one or morecommands are composed based on user input into the form element, theform element mapping to one or more portions of the one or morecommands.
 18. The system of claim 16, wherein the method furthercomprises based on the selection, causing display of a graphical userinterface comprising the one or more form elements, wherein the one ormore commands are composed from user input to the graphical userinterface.
 19. The system of claim 16, wherein the method furthercomprises: after adding the one or more commands to the query, receivinga request to modify the one or more commands added to the query; inresponse to the request, causing the one or more form elements to bepresented; and modifying the one or more commands in the query based onuser input that modifies the one or more values of the form element. 20.The system of claim 16, wherein the method further comprises, based onthe selection of the interactive region: extracting at least one valueof the one or more data items; and determining a number of the one ormore form elements to display based on the at least one value.